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Abstract 

Character education programs are school-based programs that have as one of their objectives promoting the 
character development of students. This report systematically examines the outcomes that were measured in 
evaluations of a delimited set of character education programs and the research tools used for measuring the 
targeted outcomes. The multifaceted nature of character development and many possible ways of conceptualising 
it, the large and growing number of school-based programs to promote character development, and the relative 
nearness of efforts to evaluate character education programs using rigorous research methods all combine to make 
the selection or de velopmen t of measures relevant to the evaluation of these programs especially challenging. This 
report is a step toward creating a resource that can inform measure selection for conducting rigorous, cost effective 
studies of character education programs. The report, however, does not provide comprehensive information on all 
measures or types of measures, guidance on specific measures, or recommendations on specific measures. 
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Foreword 



The National Center for Education Evaluation and Regional Assistance (NCEE) within the 
Institute of Education Sciences (1ES) is responsible for (1) conducting evaluations of federal 
education programs and other programs of national significance to determine their impacts, 
particularly on student achievement; (2) encouraging the use of scientifically valid education 
research and evaluation throughout the United States; (3) providing technical assistance in 
research and evaluation methods; and (4) supporting the synthesis and wide dissemination of the 
results of evaluation, research, and products developed. 

In line with its mission, NCEE supports the expert appraisal of methodological and related 
education evaluation issues and publishes the results through two report series: the NCEE 
Technical Methods Report series that offers solutions and/or contributes to the development of 
specific guidance on state of the art practice in conducting rigorous education research and the 
NCEE Reference Report series that is designed to advance the practice of rigorous education 
research by making available to education researchers and users of education research focused 
resources to facilitate the design of future studies and to help users of completed studies better 
understand their strengths and limitations. 

Subjects selected for NCEE Reference Reports are those that examine and review rigorous 
evaluation studies conducted under NCEE to extract examples of good or promising evaluation 
practices. The reports present study information to demonstrate the possible range of "solutions” 
so far developed. In this way, NCEE Reference Reports are aimed to promote cost-effective study 
designs by identifying examples of the use of similar and/or reliable methods, measures, or 
analyses across evaluations. It is important to note that NCEE Reference Reports are not meant to 
resolve common methodological issues in conducting education evaluation. Rather, they present 
information about how current evaluations under NCEE have focused on an issue or selected 
measurement and analysis strategies. Compilations are cross-walks that make information buried 
in study reports more accessible for immediate use by the researcher or the evaluator. 

This NCEE Reference Report is intended to help researchers identify and select measures for 
assessing the outcomes of character education programs. Character education programs are 
school-based programs that have as one of their objectives promoting the character development 
of students. This report systematically examines the outcomes that were measured in evaluations 
of a delimited set of character education programs and the research tools used for measuring the 
targeted outcomes. The multi-faceted nature of character development and many possible ways of 
conceptualizing it, the large and growing number of school-based programs to promote character 
development, and the relative newness of efforts to evaluate character education programs using 
rigorous research methods all combine to make the selection or development of measures relevant 
to the evaluation of these programs especially challenging. This report is a step toward creating a 
resource that can inform measure selection for conducting rigorous, cost effective studies of 
character education programs. The report, however, does not provide comprehensive information 
on all measures or types of measures, guidance on specific measures, or recommendations on 
specific measures. 
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EXECUTIVE SUMMARY 



Children’s social and moral development has long been a central goal of American schools 
(McClellan 1999). Through the Partnerships in Character Education Program (PCEP), located in 
the Office of Safe and Drug-Free Schools (OSDFS) in the U.S. Department of Education, the 
federal government has distributed up to about $25 million annually in grants to state and local 
education agencies for the design and implementation of character education programs. 
Conducted at the request of OSDFS and under the auspices of the Institute of Education Sciences 
(IES), the present study has three objectives: (1) to document the constructs measured in studies 
of a delimited group of character education programs; (2) to develop a framework for 
systematically describing and assessing measures of character education outcomes; and (3) to 
provide a resource for evaluators to help identify and select measures of the outcomes of 
character education programs. 



METHOD 

We approached the selection of programs for review so as to ensure inclusion of programs 
addressing the goals of PCEP and that were diverse along some key programmatic dimensions. 
We drew on three primary sources: (1) The IES What Works Clearinghouse (WWC) 2007 
review of character education programs (WWC 2007); (2) research-driven guides to character 
education developed by the What Works in Character Education Project (WWCEP), a 
collaborative effort of the Center for Character and Citizenship at the University of Missouri-St. 
Louis and the Character Education Partnership (Berkowitz and Bier 2006a, 2006b); and (3) 
grantee reports from state and local education agencies that received funds from PCEP between 
2003 and 2007. From the pool of 68 programs identified from these sources, we randomly 
selected 36 programs for review after stratifying by source, grade level of focus, and whether the 
program is comprehensive (that is, fully integrated into the life of a school) or modular (that is, a 
stand-alone program). Random selection of the 36 programs for examination ensured that the 
analysis of outcome measurement was conducted for a subset of the 68 programs which reflected 
the diversity in measured and unmeasured attributes of the larger set of 68 programs. We then 
systematically identified the studies of each program, using Psychinfo and gray literature 
searches, and focused on those studies that provided the greatest detail on outcome measurement. 
We then reviewed these studies, and developed a classification system to group related outcome 
constructs conceptually. This taxonomy, outlined in Table 2, was structured to organize 
outcomes from broad conceptual categories to increasingly specific conceptual categories. The 
broadest level distinguished between student-level outcomes and “other” level outcomes, with 
this latter category including teacher, school, parent, and community outcomes; the mid-range of 
specificity distinguished between student affective, behavioral, and cognitive outcomes; and 
finer levels of specificity distinguished, for example, between conceptual categories such as 
student knowledge and reasoning, and prosocial and risk behaviors. In reviewing studies, we 
identified all reported outcomes measured and classified them according to our taxonomy 
(Appendix B provides a crosswalk between the taxonomy and the programs selected for the 
report), we described the measures used including their psychometric properties, and we 
provided citations for the infonnation on measures. 
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KEY FINDINGS 



Research on the selected character education programs addresses a wide variety of 
outcomes. Student-level outcomes are measured in studies of 34 of 36 programs, with 25 of 
36 programs addressing one or more cognitive outcomes, 28 addressing one or more affective 
outcomes, and 31 addressing one or more behavioral outcomes. Among these student level 
outcomes, those most often measured were: academic content (measured for 14 programs), 
prosocial dispositions and interpersonal strengths (each measured for 1 1 programs), discipline 
issues and interpersonal competencies (each measured for 13 programs), and substance use and 
intrapersonal competencies (each measured for 1 1 programs). In tenns of outcomes at other 
levels (that is, beyond the student), research on 7 programs addressed teacher- level outcomes, 16 
addressed school-level outcomes, and 14 addressed parent/community-level outcomes. Staff 
morale, school climate, and parent participation in school were the constructs measured most 
often in these respective domains (for 6, 16, and 1 1 programs, respectively). 

Measurement methods were also diverse. Appendix A provides detail, by program, on every 
measure used in the studies reviewed for this report. For each program, the appendix provides a 
brief description of the program, descriptions of each measure, and an indication of which 
outcome constructs from the taxonomy each measure addressed. As shown in these tables, 
researchers employed direct and indirect assessment, as well as surveys with reports by teachers, 
parents, and students. They reported outcomes on scales and for stand-alone items, as well as 
non-scaled measures, such as attendance or disciplinary infractions. 

Table 3 summarizes information on all of the scaled measures included in the studies 
reviewed. For each measure, the table shows the name of the instrument, whether it was 
developed for the study or is an “off the shelf’ measure, its source, the type of assessment (for 
example, direct assessment versus self report), the domain it assessed (student [cognitive, 
affective or behavioral] or “other”), and a rating of its reliability. Table 4 provides a crosswalk 
between the taxonomy outlined in Table 2 and the scaled measures identified in our review with 
reported reliability of .70 or greater. 1 Our assessment of the characteristics of the scaled 
measures revealed two central themes: 

• Among the 95 scales that researchers applied in the studies reviewed here, 46 were 
developed for the study under review. An additional 17 were adapted from existing 
measures; and 32 were available “off the shelf,” having been developed and 
published through other research. Among this last category, six scales were employed 
in research on more than one of the programs under review. 

• Reporting of psychometric properties of character education outcome measures is 
not consistent. Researchers reported reliability statistics for 62 of the 95 multiple- 
item scales applied in the studies under review, with 30 of these exhibiting reliability 
of .70 or above, 27 exhibiting mixed reliability across contexts, and 5 exhibiting less 
than .70 reliability. For 33 of the 95 scaled measures, no reliability statistics were 
reported. Validity of measures was addressed less often than reliability; the research 



1 See Sattler (2001) on the choice of 0.70 as a threshold of acceptable reliability of a scale measure. 




on just 5 of the 36 programs selected provided information on the validity of the 
measures. 



CONSIDERATIONS WHEN USING THIS REPORT 

The evidence developed from studies of the sample of programs reviewed here suggests that 
character education researchers use this report’s information on outcome measurement with the 
following considerations in mind. First, the taxonomy presented here suggests a diverse array of 
outcomes may be affected by character education programming. Reference to a clear theory of 
how program elements are li nk ed to specific outcomes may help researchers to identify those 
outcomes that the program in question is most likely to affect. In the absence of a clearly 
articulated theory, researchers could “work backward” from the taxonomy presented here to 
assess the extent to which each of the constructs are likely to be influenced by their intervention, 
selecting for measurement those that seem most appropriate. 

Second, given the complexity of “character” as a construct, it could be beneficial for 
researchers to select or develop measures with demonstrated reliability and validity. While the 
measures presented here are not necessarily representative of the universe of research on 
character education programs, nor are they necessarily the best measures available, this report 
provides information on a variety of outcome measures with demonstrated psychometric 
properties. Related to this, the field of character education could benefit from more consistent 
reporting on the psychometric properties of outcome measures. Studies provided insufficient 
information to assess measures’ reliability in the case of 33 of 95 scaled measures identified 
here. Consistent reporting of measures’ psychometric properties would support comparison of 
outcomes across programs and populations and potentially improve our understanding of 
effective character education practices. 

Finally, the findings of this report highlight the importance of alignment between the 
conceptualization and measurement of outcomes. Our review revealed two ways in which 
measurement methods demonstrate a potential lack of such alignment: (1) there may be 
misalignment between items in a particular scale (they do not “hang together”); and (2) there 
may be a mismatch between the domain or construct a measure actually captures and the domain 
or construct the researcher conceptualizes or reports. Clear conceptualization of constructs and 
alignment with measures may be supported by reference to the outcome taxonomy and related 
measures presented here. 
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I. INTRODUCTION 



Children’s social and moral development has long been a central goal of American schools 
(cf. McClellan 1999). In recent years the issue has garnered renewed attention from policy 
makers, practitioners, and researchers. In 1994, the Partnerships in Character Education Program 
(PCEP) was established within the Department of Education. Over the ensuing years, PCEP 
expanded and during Fiscal Years 2003 - 2007, Congress appropriated approximately 
$25 million annually in grants to state and local education agencies for the design and 
implementation of character education programs. Concurrent with these developments at the 
federal level, in 2007, 14 states had mandated character education in their public schools and 
another 14 had enacted legislation to encourage character education in the schools (Roth-Herbst 
et al. 2007). 

This study examines outcome measures used in the evaluation of character education 
programs. The impetus for the study was a request to the Institute of Education Sciences (IES) 
from the Office of Safe and Drug-Free Schools (OSDFS), which administers PCEP, to provide 
guidance on outcome measurement for PCEP grantees. All PCEP grantees are required to 
conduct evaluations of their programs under the terms of their grants. The purposes of this study 
are to describe a framework for conceptualizing character education outcome measures and to 
serve as a resource to help PCEP grantees and other evaluators leam about and select measures 
for evaluating character education programs. 

The programs intended to promote character development are numerous and diverse. From 
the early 1990s to the present, the number of organizations developing and disseminating 
character education curricula has grown dramatically (Damon 2005). Both for-profit and 
nonprofit organizations are prominent among character education program developers and their 
approaches range from discrete, modular curricular programs to comprehensive, whole-school 
reform efforts. The range of intervention elements reflects the widely varying conceptions of 
what can or should be taught as “character education.” Indeed, among researchers there has been 
a tendency in recent years to consider as character education virtually any program that seeks to 
promote positive youth development (Berkowitz 2007). 

The What Works Clearinghouse 2 (WWC) conducted a review of character education 
programs in 2007 to identify programs and studies of programs, evaluate the quality of causal 
evidence on each program’s effects, and document the effects of programs based on studies that 
met WWC evidence standards. While WWC researchers recognized the broad and varied nature 
of character education, they articulated and applied specific criteria in selecting programs for 
review to maintain comparability across interventions. In particular, WWC considered for review 



2 The Department of Education established the What Works Clearinghouse to serve as a central, authoritative 
source of scientific evidence on the effectiveness of education interventions. See the WWC website at 
http://ies.ed.gov/ncee/wwc/ for more information. 
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only those “programs that deliberately attempt to develop students’ character by teaching core 
values and that had most if not all of their lesson plans or prescribed activities directly related to 
instilling those values” (WWC 2007). 

The variety of programs and their diverse goals lead to a corresponding diversity in the way 
the outcomes of character education programs are conceptualized and measured. The WWC 
review classified student outcomes into three broad categories: (1) cognitive (understanding and 
being able to reason about character concepts as well as academic content), (2) affective (relating 
to attitudes, emotions, motives, and beliefs about what is important), and (3) behavioral (acting 
on understandings and beliefs). Other researchers have developed detailed classifications of more 
specific constructs that character education interventions are hypothesized to affect. Prominent 
among these, Peterson and Seligman (2004) worked with other scholars in the field to develop 
the “Values in Action” (VIA) classification, which seeks to delineate those traits that comprise 
the multidimensional construct of character across populations. Corresponding to the VIA 
classification, these same researchers have developed instruments to measure the six strengths — 
wisdom, courage, humanity, justice, temperance, and transcendence — and 24 positive traits 
identified in their model. Branching off from Seligman and Peterson, other developmental 
scientists have put forth the “five Cs” as a model of positive development: competence, 
connection, character (comprised of VIA’s six strengths), confidence, and caring or compassion 
(Lerner et al. 2000); and scholars have worked to develop or compile indicators to measure 
outcomes in each domain (Roth-Herbst et al. 2007). Focusing specifically on positive aspects of 
character, the Josephson Institute has identified six “pillars” — trustworthiness, respect, 
responsibility, fairness, caring, and citizenship — which form the basis of Character Counts!, a 
nationally available character education program (Josephson Institute 2009). A model central to 
many current character education efforts includes Lickona and Davidson’s (2005) “8 Strengths 
of Character” which promotes developmental outcomes such as critical thinking, diligence, 
social-emotional skills, ethical thinking, commitment to moral action, self-discipline, community 
involvement, and pursuing a life with noble purpose. Still, citing the VIA as an exception, Roth- 
Herbst, Borbely, and Brooks-Gunn (2007) have concluded that “little work... has attempted to 
create a reliable and valid scale to measure the many components implied by the term character” 
(p. 175). 

Even as the field continues to evolve, there is a pressing need to identify and measure the 
outcomes of character education programs. For example, where character education programs 
are required (as is the case in at least 14 states), accountability requirements dictate that they 
must demonstrate outcomes (Roth-Herbst et al. 2007). Similarly, recipients of PCEP grants are 
required to conduct evaluations. The 2009 PCEP request for application lists factors that may be 
considered in evaluating programs, including discipline, student academic achievement, and 
school climate, among other factors, and establishes a competitive preference for projects that 
use an experimental or quasi-experimental design, noting the importance of valid and reliable 
measurement of outcomes (U.S. Department of Education 2009). 

In this context, the present study addresses three objectives: (1) to document the constructs 
measured in studies of a delimited set of character education programs; (2) to develop a 
framework for systematically describing and assessing measures of character education 
outcomes; and (3) to provide a resource for evaluators to help identify and select measures of the 
outcomes of character education programs. To ensure that the study reflects the experiences of 
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recent PCEP grantees and responds to the desire of OSDFS to provide guidance for future 
grantees, the study includes character education programs implemented by recent PCEP 
grantees, as well as programs identified from two reviews of school-based character education 
programs. However, a comprehensive investigation of the measures used to study character 
education programs was beyond the scope of this study. The report aims both to bring together in 
one place information on a set of outcome measures and to provide a framework for reviewing 
measures that individual researchers are considering for use in assessing the effects of a 
particular character education program. 
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II. METHOD 



To address this study’s objectives, we first identified a group of character education 
interventions for which a systematic examination of outcome measurement would be conducted. 
Our goal was to include programs incorporating a broad array of elements and seeking to address 
multiple outcomes. However, we needed to restrict the number of programs examined in light of 
resource constraints. Further, we sought to select programs in an objective and replicable 
manner. The program identification and selection process was designed to allow us to examine a 
set of programs and the available research on these programs, while ensuring that the nature and 
quality of the selected programs and studies would be broadly representative of the larger set 
from which the study sample was drawn. Next, using a taxonomy for classifying character 
education outcomes — which was developed from the literature and amended as our review 
revealed a need for different categories — we conducted a systematic review of research studies 
on the selected programs to identify and classify the outcomes measured. We then conducted 
cross-case analysis to identify the approaches to measurement and to describe the characteristics 
of the measures used to assess outcomes. 



A. DEFINING PROGRAMS OF INTEREST 

Defining the Set of Programs of Interest. Recognizing that the tenn “character education” 
may be used in reference to a large number of diverse interventions, we approached the selection 
of programs for review in a manner intended to ensure that a broad variety of relevant programs 
would be included. In assembling a set of programs for consideration, we used the What Works 
Clearinghouse (WWC) review of character education interventions as a starting point (WWC 
2007). As noted, the WWC protocol for the character education review defined character 
education programs as “programs that deliberately attempt to develop students’ character by 
teaching core values and that had most if not all of their lesson plans or prescribed activities 
directly related to instilling those values.” WWC’s systematic search yielded 41 programs 
meeting this definition. WWC researchers then reviewed the quality of the available research 
evidence on the impacts of each on student outcomes. This process yielded two groups of 
programs: 13 with at least one study meeting WWC evidence standards for measuring causal 
program effects and 28 with no studies that met WWC evidence standards. Because our central 
interest is in outcome measurement, rather than impacts, we included both of these groups in the 
set of programs considered for this study. (It should be noted, however, that among the 28 
programs not meeting WWC evidence standards, no evaluations were available for 14 of them, 
leaving just 14 candidates for our review.) 

As noted, the WWC adopted a definition of character education that maintained 
comparability across interventions, as was appropriate for a comparison of program impact 
estimates. However, this definition excluded programs that are widely intended and considered 
to provide character education. To incorporate a broader conception of character education, we 
turned to the What Works in Character Education Project (WWCEP — not affiliated with the 
What Works Clearinghouse). WWCEP is a collaborative research effort of scholars at the Center 
for Character and Citizenship at the University of Missouri-St. Louis and the Character 
Education Partnership, a national umbrella organization focused on advancing character 
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education in America’s elementary and secondary schools. In compiling two “research-driven 
guides” focusing specifically on the outcomes of character education (Berkowitz and Bier 2006a, 
2006b), WWCEP researchers generated a conceptual model of character education that led them 
to consider “any school-based K-12 initiatives either intended to promote the development of 
some aspect of student character or for which some aspect of student character was measured as 
a relevant outcome variable” (Berkowitz and Bier 2006b, p. 3). Following a systematic research 
protocol similar to WWC’s (but using less stringent evidence standards), WWCEP researchers 
ultimately identified 33 programs for which scientific research was available, which included 
drug and alcohol prevention, violence prevention, service learning, and social-emotional learning 
programs. These 33 interventions comprised a second group of programs considered for the 
present study. 

Character education programs funded by the PCEP are a third source of programs 
considered for this study. PCEP is authorized to make grants to state and local education 
agencies for the development, implementation, and evaluation of character education programs 
that “are able to be integrated into classroom instruction and to be consistent with State academic 
content standards; and... able to be carried out in conjunction with other educational reform 
efforts” (U.S. Department of Education 2002). The legislation cited several examples of 
character elements that grantee programs might seek to promote: caring, civic virtue and 
citizenship, justice and fairness, respect, responsibility, trustworthiness, giving, and “any other 
elements deemed appropriate” by the grantee. Similarly, the federal legislation offered several 
examples of outcomes that might be measured in evaluating grantee programs, including: 
discipline issues, academic achievement, participation in extracurricular activities, parental and 
community involvement, faculty and administration involvement, student and staff morale, and 
school climate. Between 2003 and 2007, PCEP awarded grants to 58 state and local education 
agencies around the country. The 21 programs covered by grants for which the grantee had 
completed at least one report to PCEP by summer 2008 formed the third group of programs 
considered for review here. 

A total of 68 unique character education programs for which at least one report was 
available were identified from these three sources (WWC, WWCEP, and PCEP). From this set of 
68 programs, we selected 36 for intensive review of outcomes measured and measurement 
methods in available research on the programs. 



B. SELECTING PROGRAMS FOR REVIEW 

A preliminary review of research on character education programs indicated that studies of 
such programs included a large, varied set of measures. Our goal in selecting programs for 
intensive review was to create a sample that would be diverse along key programmatic 
dimensions, and thus be likely to reflect the full range of outcomes of potential interest. It was 
also important that the sample selection criteria be replicable. 

Before selecting programs, we scanned published and unpublished studies, as well as 
developer and evaluator websites to obtain basic descriptive information on each program and, to 
the extent available, on its intended outcomes. Based on this preliminary examination of 
information on the programs and their intended outcomes, we judged that a sample of 
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36 programs would be sufficient to reflect the diversity of programs in our set of 68. We also 
judged that an examination of 36 programs was consistent with the resources available. 

First, we stratified programs based on the four groups included for consideration — that is, 
13 programs meeting WWC evidence standards, 14 programs not meeting these standards, 
33 WWCEP-identified programs, and 21 PCEP grantees. There was some overlap between the 
groups (11 programs were identified by more than one source), and we allowed programs 
appearing in more than one group to have a higher probability of selection. Second, to portray 
the broadest range of outcome measures, it was important to include programs designed for 
different grade levels. So we classified programs according to whether the program was designed 
for all grades (K-12), lower grades (K-6 or K-8), or higher grades (middle and/or high school). 
Third, following the WWC approach, we categorized programs by type as either comprehensive 
or modular. Comprehensive programs are aimed at affecting the school as a community by 
integrating character education into the full spectrum of school activities. Modular programs are 
designed to be used in a single classroom or group of classrooms or to involve a particular type 
of event or activity, such as an inspirational speaker, which can be a school-wide event. 

Our selection process assigned a more than proportionate share of the sample to programs 
reviewed by WWC because we had all studies on these interventions and we judged the studies 
were of sufficient quality to support investigation of outcome measurement. In contrast, much of 
the research from WWCEP and all of the PCEP program research was unpublished and therefore 
potentially difficult to obtain (requiring direct requests from authors), and the quality of 
information on outcome measurement was unknown. We randomly selected 10 programs from 
each of the two WWC groups, and eight programs each from the WWCEP and PCEP groups, for 
a total of 36 programs from a sample frame of 68 different programs. The lists of selected 
programs were then visually inspected to ensure the selection resulted in a sample with sufficient 
variation on grade category and comprehensive/modular type. While this selection process was 
not designed to ensure that all character education programs were represented, it did achieve 
broad representation of programs and studies in character education and provided clear, 
replicable criteria for selection of the interventions subject to examination. 



C. APPROACH TO ANALYSIS 

Sources Reviewed. In line with our goal of ensuring this study examined a diverse set of 
programs and the outcomes these programs aimed to affect, we selected for examination the 
study or studies of each selected program that together provided comprehensive information on 
program elements, outcomes, and outcomes measurement. In the case of PCEP grantee reports, 
the report itself was the focal study. For WWC and WWCEP programs, we analyzed 
bibliographies from the two groups’ respective reviews, and gave preference to studies that (1) 
clearly explained outcome measurement, (2) were published in the past 20 years (so as to reflect 
a relatively current version of the intervention), and (3) appeared in peer-reviewed journals. We 
also reviewed any available developer and evaluator websites to obtain infonnation on program 



3 Decision rules for repeated sampling had been determined prior to selection to ensure diversity in these 
characteristics, but the first selection yielded a satisfactory mix. 
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elements. Finally, to ensure that we had access to the most recent research on each program, we 
conducted a search for each program in Psychlnfo, an on-line database of psychological research 
conducted from the 1800s to the present (our search focused on the period since 1988). For 16 of 
the 36 programs selected for review, the only research available at the time was a single 
unpublished study. Table 1 lists the programs that were ultimately reviewed for this study, along 
with the grade levels for which each was designed, and a designation of comprehensive or 
modular. The designations of comprehensive or modular are usually, but not always, mutually 
exclusive, as some programs make different versions available; 16 programs are modular, 15 are 
comprehensive, and five can be implemented in either way. Appendix A provides information on 
the studies reviewed for each program. 

Classification of Outcome Constructs. To examine measurement across the 36 programs, 
it was necessary to create a structure allowing conceptually similar outcomes to be grouped 
together. Development of the taxonomy used in this study was iterative. We started with a set of 
categories drawn from the literature; these categories were refined as review of the research on 
the 36 selected programs revealed a need for more or less specificity in particular areas. 

Specification of the taxonomy began by integrating outcome classification approaches from 
each of the sources used to identify our programs (WWC, WWCEP, and PCEP). 4 Both the 
WWC and WWCEP begin with a basic affective-behavioral-cognitive distinction, which we 
adopted. The cognitive domain refers to the student’s understanding of and ability to recognize 
values and how values may affect people and actions in different situations. The affective 
domain refers to caring about values, and encompasses attitudes, feelings, and self-perceptions 
that are related to character or the values and behaviors associated with it. The behavioral 
domain encompasses all outcome variables pertaining to the enacted behavior of students, 
including displays of core values (honesty, fairness), prosocial behavior (support for peers), or 
decreased problem behavior. The outcome taxonomy used in this study, shown in Table 2, 
includes much greater detail on the constructs within each of the three main domains than was 
considered by the WWC. The categories within broad domains are drawn primarily from the 
classification developed by WWCEP, which was designed around the specific constructs 
identified by Berkowitz and Bier (2006a). 

Finally, while WWC and WWCEP classifications focused exclusively on student outcomes, 
the legislative language authorizing the PCEP grants for the development and implementation of 
character education programs (U.S. Department of Education 2002) included guidance directing 



4 For more information on the WWC taxonomy, see the WWC Evidence Review Protocol for Character 
Education Interventions at http://ies.ed.gov/ncee/wwc/. For more information on WWCEP taxonomy, see Berkowitz 
and Bier (2006a). For information on outcome areas identified by PCEP, see legislation regarding grant 
competitions for character education programs in the Elementary and Secondary Education Act of 1965, as 
amended, Title V, Part D, Subpart 3, Sec 5431 — Partnerships in Character Education', 20 U.S.C. 7247 (available 
at http://www.ed.gov/programs/charactered/legislation.html). 
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TABLE 1 



CHARACTERISTICS OF PROGRAMS SELECTED FOR REVIEW 



Comprehensive/ 

maProgram Name Grades Modular Source 



K-6 M WWC-ne 

6-12 M WWC, WWCEP 



AEGIS 

Building Decision Skills (BDS) 

Building Esteem in Students Today (BEST) 

Caring School Community (CSC; formerly Child 
Development Project) 

Changing Lives (CL) 

Character Counts! (CC!) 

CHARACTERplus (CP) 

Character Quality Program (CQ) 

Community of Caring (C of C) 

Connect with Kids (CWK) 

COOL Kids 

Educating for Character (E for C) 

Giraffe Heroes (GH) 

Heartwood Ethics Curriculum for Children (HECC) 

I Can Problem Solve (ICPS) 

Institute for Character Education (ICE) 

Just Communities (JC) 

Learning for Life (LFL) 

LIFT 

Lions Quest - Skills for Action (LQ Skills for Action) 
Lions Quest - Skills for Adolescence (LQ Skills for 
Adol) 

Open Circle (OC) 

Partnerships in Character Education (PCE) 

Positive Action (PA) 

Project ESSENTIAL (PE) 

Project Heart, Head, Hands (H3) 

Raising Healthy Children (RHC) 

Resolving Conflicts Creatively Program (RCCP) 
Social Competence Promotion Program for Y oung 
Adolescents (SCPP-YA) 

Social Decision Making/Problem Solving (SDM/PS) 
Teen Outreach (TO) 

Too Good for Drags and Violence (TGFDV) 

Too Good for Violence (TGFV) 

Topeka Character Education Initiative/Character First 
(Topeka Char. 1st) 

Tribes TLC 
Voices LACE 



K-8 


C 


WWC-ne 


K-6 


C 


WWC, WWCEP, 
PCEP 


K-12 


M 


WWC-ne 


PK-12 


C/M 


WWC-ne 


K-12 


C 


PCEP 


PK-12 


c 


PCEP 


K-12 


c 


WWC-ne 


3-12 


M 


WWC 


K-9 


c 


PCEP 


K-12 


c 


PCEP 


K-12 


C/M 


WWC-ne 


PK-6 


M 


WWC 


PK-6 


M 


WWCEP 


K-12 


C 


PCEP 


9-12 


C 


WWCEP 


K-12 


M 


WWC-ne, WWCEP 


K-5 


M 


WWCEP 


9-12 


C 


WWC 


6-8 


C 


WWC, WWCEP 


K-5 


M 


WWC-ne, WWCEP 


6-12 


C/M 


PCEP 


K-12 


C/M 


WWC, WWCEP 


PK-7 


M 


WWC-ne, WWCEP 


K-6 


M 


PCEP 


K-12 


C 


WWCEP 


K-12 


C 


WWCEP 


6-9 


M 


WWCEP 


K-8 


M 


WWCEP 


7-12 


M 


WWCEP 


9-12 


M 


WWC 


K-8 


M 


WWC 


K-12 


C 


PCEP 



K-12 C WWC-ne 

K-12 C/M WWC 



Note: WWC=What Works Clearinghouse; WWC-ne=What Works Clearinghouse-no evidence; WWCEP=What 

Works in Character Education Project; PCEP=Partnerships in Character Education Programs. 
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grantees to consider outcomes for teachers or school staff, schools, and parents or communities, 
as well. Accordingly, we expanded the classification system to include these other outcomes: 
teacher- or administrator-level outcomes, school-level outcomes, and parent- or community-level 
outcomes. 

Table 2 shows the outcome classification as refined after completing reviews of the research 
on the 36 selected programs. While this classification is intended to be comprehensive enough to 
account for the many constructs measured as outcomes in the character education research 
reviewed here, it nevertheless represents a simplification of some of the key outcomes that 
character education seeks to affect. In particular, the assignment of constructs to the affective, 
behavioral, or cognitive domain does not fully recognize the complex, multifaceted nature of 
some developmental outcomes. For example, it is widely agreed that constructs such as self- 
esteem or self-efficacy involve affective, behavioral, and cognitive elements (Bandura 1997; 
Harter 1990). We sought to reflect such complexity in our analysis and reporting by categorizing 
some instruments as measuring constructs in multiple domains. Further, we note that the nature 
of the measures in the studies reviewed has led us to classify certain constructs differently from 
other researchers. Finally, our taxonomy of character education outcomes is not exhaustive: 
character education programs not included in our review may measure constructs that we have 
not considered. 

Research Review. Our review process included three major phases. First, one of four 
researchers reviewed the selected study or studies for nine programs, using a review guide 
designed to collect information on program elements, fidelity and intensity of implementation, 
all outcomes measured, instruments used, and citations for measures or instruments. Within this 
rubric, the researcher noted any psychometric infonnation provided for the measures. We were 
particularly interested in the reliability and validity of outcome measures, which we discuss later. 
If the study cited other research in describing the measure or instrument, the researcher sought to 
locate the citation and record pertinent information. In the second phase of our review, the 36 
selected programs were redistributed among the four members of our research team; each 
researcher reviewed the materials assembled for three programs initially reviewed by each of the 
other team members (for a total of nine programs reviewed on the second round). In this way, the 
information on each program was reviewed by two researchers across the two review rounds. 
The second review step served as a quality control check on the reviews performed on the first 
round and an analytic step in which each researcher distilled the information contained in the 
review rubrics into the analytic tables that appear in this report and Appendices A and B. During 
this process, research meetings were conducted at least weekly to ensure consistency among 
reviews and to reconcile any differences of interpretation between the reviewers of each study. 
Finally, in the third phase of the process, we conducted cross-program analysis to identify 
patterns in the research under review. This phase involved review by the same four researchers 
of the analytic tables from Appendices A and B and further distillation into the tables appearing 
in the body of this report. 
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TABLE 2 



TAXONOMY OF CHARACTER EDUCATION OUTCOMES 





Student-Level Outcomes 


Cognitive 




Behavioral 


Knowledge 




Competencies/ski l Is 


• understanding values/norms 




• resistance 


• risk prevention 




• responsibility 


• interpersonal knowledge 




• integrity 


• intrapersonal knowledge 




• respect 


• academic content 




• leadership 


Reasoning 




• intrapersonal competency (self-control, self- 




discipline, self-regulation) 


• moral/ethical reasoning 




• interpersonal competency 


• critical thinking/decision making 




• communication 


Affective 




• coping 


Attitudes/Motives 




Prosocial behaviors 


• prosocial dispositions 




• service 


• attitudes toward school 




• healthy lifestyle 


• attitudes toward risk/health 




• kindness 


• civic dispositions 




• trustworthiness 


• attitudes toward diversity 




• justice, fairness 


• intrapersonal strengths (self-esteem, self-efficacy) 

• internalizing problems 


• positive participation 
Risk behaviors 


Attitudes/Emotions 




• substance use 


• caring (e.g., empathy) 




• sexual risk-taking 


• reflectivity 




• violence 


• school bonding/school engagement 




• absence/tardiness 


• justice, fairness 




• discipline issues 

• crime 



Other Outcomes 



Teacher/Admin-Level Outcomes 

• Knowledge of child development 

• Support (endorse or value what goes on in school) 

• Understand values/norms 

• Attendance 

• Staff morale 

• Professional efficacy 

Parent-Community Level Outcomes 

• Parenting skills 

• Participation in school 

• Parent support of school/program 

• Community climate/environment 



School-Level Outcomes 

School climate 

• collective norms/values 

• interactions among students, staff, parents 
(positive/ negative content) 

• inclusion (e.g., individuals made to feel part of 
the school) 

• Social systems (advisories, leadership teams, 
buddies, vertical families) 

Democratic governance practices 
Positive leadership 
Positive physical environment 
Positive academic environment 
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III. FINDINGS 



A. OUTCOMES ASSESSED IN STUDIES OF CHARACTER EDUCATION 

PROGRAMS 

To understand what constructs the studies of selected character education programs are 
addressing, we examined researchers’ statements about what outcomes they were measuring, 
reported findings, and available infonnation on the measures themselves. 

Figures 1-3 display which constructs are being measured in studies of the 36 selected 
character education programs. Each figure shows the number (in parentheses) of character 
education programs for which our reviewed studies included measurement of the corresponding 
constructs from our classification. (Appendix A provides detailed information on each selected 
program and on the constructs and measures used in evaluations of that program. Appendix B 
summarizes which constructs in Table 2 are measured for each program.) Figure 1 addresses 
outcomes at the broadest level of the taxonomy, and shows that student-level outcomes are 
measured most often, in studies of 34 of 36 programs. At the other broad levels, research on: 7 
programs addressed teacher-level outcomes, 16 addressed school-level outcomes, and 14 
addressed parent/community-level outcomes. 

Examining constructs at the lower (that is, more specific) level of the taxonomy, the wide 
bars in Figure 2 show that student-level cognitive outcomes are addressed in the research on 
25 of 36 programs, student affective outcomes are addressed for 28 programs, and student 
behavioral outcomes are addressed for 31 programs. 5 Figure 2 shows great variety in the 
constructs measured, calling attention to the relative frequency with which different outcomes 
are assessed in the selected studies of selected character education programs. For student-level 
cognitive outcomes, academic content, including knowledge measured by grades and test scores, 
was assessed most frequently (in studies of 14 programs). Interpersonal knowledge and 
understanding values and nonns were measured in studies of six and seven programs, 
respectively. Finally critical thinking and knowledge of risk prevention were measured least (in 
studies of one and no programs respectively). In the affective sphere, prosocial dispositions and 
intrapersonal strengths, such as self-esteem and self-efficacy, were measured most often (in 
studies of 1 1 programs each); in contrast, reflectivity and affective aspects of justice/fairness 
were assessed with little frequency (in studies of one program each). Finally, for behavioral 
outcomes, discipline issues and interpersonal competencies, such as conflict resolution and 
cooperation, were assessed most often (in studies of 13 programs each); substance use and 
intrapersonal competencies, such as self-control and self-discipline, were also measured with 



5 Due to limitations on available information, not all measures could be categorized at the lowest level of 
classification. Therefore, the numbers presented in Figures 2 and 3 may underestimate the number of programs for 
which the research studies addressed constructs at the more specific level of the taxonomy. Furthermore, the 
numbers for the broadest levels of the taxonomy include counts from all levels below, in addition to those that could 
not be classified at a lower level. 
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FIGURE 1 



NUMBER OF PROGRAMS FOR WHICH STUDENT, TEACHER, SCHOOL, AND 
PARENT OUTCOMES WERE MEASURED 



Humberof Programs 

0 5 10 15 20 25 50 35 
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School- Level Outcomes 416) 
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FIGURE 2 



NUMBER OF PROGRAMS FOR WHICH SPECIFIC STUDENT OUTCOMES 

WERE MEASURED 



Cognitive (25) 
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Internalizing problems (6) 
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FIGURE 3 



NUMBER OF PROGRAMS FOR WHICH STUDENT, TEACHER, SCHOOL, AND 
PARENT-COMMUNITY OUTCOMES WERE MEASURED 



Number of Programs 
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Support (Endorse or Value What Goes on 
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Professional Efficacy (3) 

School-Level Outcomes (16) 

School climate (16) 

Collective norms/values (9) 

Interactions among students, staff, 
parents (positive/ negative content) (14) 

Inclusion (e.g. individuals made to 
feel part of the school) (7) 

Social Systems (Advisories, Leadership 
Teams, Buddies, Vertical Families) (2) 

Democratic Governance Practices (3) 

Positive Leadership (3) 

Positive Physical Environment (1) 

Positive Academic Environment (3) 

Parent-Community Level Outcomes (14) 

Parenting Skills (3) 

Participation in School (11) 

Parent Support of School/Program (1) 
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relatively high frequency (in studies of 1 1 programs each). Conversely, healthy lifestyle was 
unmeasured and three other constructs were measured in studies of one program each 
(communication, coping, and behavioral aspects of justice/faimess). 

Figure 3 indicates that two constructs at the teacher, school, or parent level were measured 
with relatively high frequency. At the school level, school climate — the tone or nature of 
interactions among members of the school community — was measured in studies of 
16 programs. At the parent level, studies of 1 1 programs addressed parent participation in school. 
None of the teacher/administrator-level outcomes were measured with such frequency, but staff 
morale was assessed in studies of six programs. Among constructs addressed with the lowest 
frequency, at the teacher level, no studies addressed knowledge of child development and studies 
of one program addressed attendance. At the school level, the nature of social systems and the 
physical environment were measured in studies of two and one program, respectively. Finally, at 
the parent/community level, parents’ support for the school or program was measured in studies 
of one program. Community-level outcomes were only addressed in one study, although 
program materials sometimes identify the broader community as a focus (for example, Character 
Counts!, CHARACTERplus, and Topeka Character Education Initiative/Character First) and 
PCEP’s enabling legislation mentions community involvement as being of interest to program 
evaluations. 

Looking across the figures and across the constructs, several issues merit note. First, as 
stated above, the reviewed studies of character education programs focused most often on 
student outcomes. At the same time, Figures 2 and 3 also demonstrate the great breadth of 
conceptual categories that might be affected by character education programming. Still, for most 
outcomes at the teacher, school, and parent levels, it sufficed to have two levels of specificity in 
the taxonomy (indicated by the two widths of the bars). School climate provided the single 
exception. In contrast, for student outcomes, many specific outcomes were measured. 



B. MEASUREMENT METHODS AND CHARACTERISTICS OF MEASURES 

The outcome constructs in Figures 1-3 were measured using a variety of methods. 6 Both 
direct and indirect assessments were used to gauge student-level outcomes. Direct assessments 
gauge students’ ability to perform a specific task or demonstrate knowledge (for example, 
through a test of content knowledge or an achievement test). Indirect assessments ask students to 
respond to a specific scenario by describing what they would do or think in the situation 
represented. Responses are then scored by a third party against a well-specified standard. 
“Reports” by students, teachers, and parents record the respondent’s perceptions about the beliefs 
or behaviors of the individual or group being assessed. Students reported on their own beliefs 
and behaviors, as well as their perceptions of school, just as teachers and administrators reported 
on their own beliefs and behaviors, as well as perceptions of themselves and of students. 

The various assessment approaches included different types of measures, which fall into 
three broad categories: (1) scales or subscales, (2) stand-alone items, and (3) non-scaled 

6 Direct observation tools were not used in any of the studies of programs reviewed. Therefore, none are 
addressed in this report. 
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measures. Scaled measures typically take the sum or average of multiple conceptually related 
items to derive a single score or a series of subscores, each of which is meant to reflect a single 
construct or combination of constructs. Stand-alone items are single questions gauging some 
knowledge, belief or behavior. Non-scaled measures are those that are reported in natural units, 
for example, attendance or disciplinary infractions. Among all three types, study authors 
developed their own measures, or used “off the shelf’ measures and adaptations of them for the 
study at hand. 

In examining the characteristics of outcome measures we focus on scales and subscales 
because well established methods for assessing the psychometric properties of scales make them 
the method of choice for measuring many of the complex, multifaceted, and not directly 
observable outcomes character education interventions seek to bring about. Table 3 provides 
information on the 95 scaled measures employed in studies of the 36 selected programs. The 
number of scaled measures used in the studies reviewed ranged from none (in studies of 5 
programs) to 9 (in studies of 1 program), with the outcomes of 16 programs being assessed with 
one or two scales and the outcomes of 14 programs with three to six scales. For the discussion of 
Table 3, note that the unit of analysis changes from the program to the scaled measure. 

Instruments are listed by program in the first two columns on the left. For instruments 
developed by researchers for the study reviewed here, we use the researchers’ own descriptive 
tenns as the name. Where a published (“off the shelf’) instrument was used, we report that name. 
For purposes of cross-referencing, the names of these scaled measures are underlined in the 
program tables in Appendix A. Indented items in the “Instrument Name” column reflect use by 
the researchers of selected items from that instrument in combination with other items, either 
from other published measures or developed by the researchers themselves. In the “Developed 
for Study” column, we distinguish among three categories: (1) existing measures (categorized as 
“No,” not developed for the study); (2) those that are entirely new and those where selected 
items are combined with items from another instrument to fonn a single new scale (categorized 
as “Yes,” developed for study); and (3) those instruments that were “Adapted” from an existing 
measure, by reducing the number of items or rephrasing particular items. Each of the instruments 
summarized in Table 3 was designed to tap a single construct or group of constructs as these 
were defined by the researchers who conducted each study. In the column under “Domain 
Assessed,” Table 3 indicates whether an instrument tapped constructs belonging to one or more 
of the highest levels of our outcome classification — student affective, behavioral, and cognitive 
outcomes or to an “other” category, which includes teacher-, school-, parent-, and community- 
level outcomes. As indicated for those scales with more than one checkmark, a single instrument 
can assess multiple domains. 

Additionally, we report the type of sample and grade level for which each instrument was 
developed, as reported either in the program study or in the document cited by the authors in 
describing the instrument. In the “Characteristics of Development Sample” column, sample types 
are categorized as either the “Study Sample,” (that is, the measure was developed on the sample 
studied in the program report); an “External Sample” (the measure was developed using a non- 
nationally representative sample for research other than the study reviewed here); or “Nationally 
Representative” sample (the measure was developed with an external, nationally representative 
sample). This information is potentially useful for audiences interested in assessing the outcomes 
of character education programs for particular populations or grade levels. 
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In the last column of Table 3, we summarize available information on the “Reliability” of 
the measures, which refers to the capacity of an instrument to measure a construct consistently. 
While reliability can be assessed for different dimensions, three commonly used measures of 
reliability are inter-rater reliability, test-retest reliability, and internal consistency reliability. 
Inter-rater reliability reflects the degree to which different raters or assessors come to the same 
score or conclusion using a single measure. Test-retest reliability captures whether a measure 
will produce the same score if re-administered in a short period of time (that is, before 
individuals assessed by the measure have an opportunity to grow or change). Finally, internal 
consistency reliability reflects the degree to which items within a measure tap aspects of the 
same construct (that is, the degree to which items in a single measure hang together). The various 
forms of reliability are each appropriate in different situations (Sattler 2001). 

Typically, authors of the studies examined reported the estimated internal consistency of 
scale items using Cronbach’s alpha (Cronbach 1951). However, researchers sometimes reported 
one of the other reliability estimates (indicated in Table 3 by the symbol f). Sometimes authors 
reported multiple reliability statistics, for example, across subscales of a single instrument, 
among different subpopulations, or among different reporters (students, parents, teachers). To 
simplify presentation, we have rated the various reliability indicators that studies provided or that 
we identified in other sources on a single, three-point scale. In the “Reliability Rating” column of 
Table 3, a “2” indicates that the instrument was reported to have a reliability (of any type) equal 
to or exceeding .70 (a typical threshold at which scales are considered to demonstrate adequate 
reliability) 7 ; a “1” indicates reported reliability less than .70; and “0” indicates that there was 
insufficient information provided to gauge the measure’s reliability. Where a range is given 
(1-2), this indicates that different levels of reliability were reported. An asterisk indicates that 
reliability information was available only from a source other than the study sample (that is, the 
authors of the study under review provided no reliability information for their own sample, but 
cited another source — typically the instrument developer — from which reliability information 
was available). 

Validity is a second key psychometric property of measures employed in studies of character 
education programs. Validity refers to the degree to which a measure captures the construct it 
was designed to measure (Sattler 2001), with different form of validity taking a different 
approach to detennining the extent to which the content of a measure aligns with an alternative 
indicator of that construct. Because studies of only 5 of 36 programs addressed any type of 
validity — and the information provided by these was incomplete — the results on validity are not 
summarized in Table 3. However, we discuss the validity of measures later in this chapter. 



7 See, for example, Sattler 2001 
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TABLE 3 



OVERVIEW OF SCALED INSTRUMENTS USED IN STUDIES OF 36 CHARACTER EDUCATION PROGRAMS 







Developed 
for Study 6 




Type of 
Assessment 6 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


AEGIS 


(no scale reported) 


















BDS 


Janis-Field Feeling of 
Inadequacy Scale 


No 


Robinson and 
Shaver 1973 


Student report 


X 






External 
sample, grades 
NR 


1-2*1 


BDS 


Social Relatedness Scale 
Social Development 
Scale — Non-School 
Responsibility, School 
Responsibility, Future 
Political Participation 
and Future Affiliation 
Subscales 


Adapted 


Newmann and 
Rutter 1983 


Student report 


X 


X 




External 
sample, grades 
9-12 


1-2* 


BEST 


BEST Student Survey of 
School Connectedness 


No 


BEST 2000 


Student report 


X 


X 




NR* 


0 


BEST 


Social Skills Rating 
System — Social Skills 
Scale 


No 


Gresham and 
Elliot 1990 


Student report 


X 


X 


X 


Nationally 
representative, 
grades 3-12 


1-2* 


CSC 


Intra/Interpersonal 

Competency 

Child Loneliness 
Scale 

Social Anxiety Scale 
for Children 


Yes 


Asher and 
Wheeler 1985 
LaGreca, 
Dandes, Wick, 
Shaw, and Stone 
1988 


Student report 


X 


X 




Study sample, 
grades 3-6 


0 


CSC 


Enhance Relationships 
Survey 


No 


Developmental 
Studies Center 
n.d. 


Teacher report 








X Study sample, 

grades NR 


1-2 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 


Domain Assessed* 




Characteristics 


Reliability 

Rating 11 


Program 11 


Instrument Name b 


Citation/s d 


Affect Behav Cognit 


Other 


of Development 
Sample 8 


CSC 


Sense of Community 
Survey 


No 


Developmental 
Studies Center, 
see Solomon, 
Battistich, and 
Horn, 1996 


Student report 


X 




Study sample, 
grade 3 


1-2 


CSC 


California Achievement 
Test, Sixth Edition 
(CAT/6) /TerraNova, The 
Second Edition 


No 


CTB/McGraw- 
Hill n.d. (a) 


Direct assessment 


X 




Nationally 
representative, 
grades 3 and 7 


0 


CSC 


California Standards Test 


No 


California 
Department of 
Education n.d. 


Direct assessment 


X 




External 
sample, grades 
2-12 


0 


CL 


Impressions of the School 
and Interaction Patterns 


Yes 


Katsuyama and 
Kimble 2002 


Teacher Report 




X 


Study sample, 
grades K-6 


0 


CL 


Behavioral Expectations 
at School 


Yes 


Katsuyama and 
Kimble 2002 


Student report 




X 


Study sample, 
grades K-6 


0 


CL 


Student Behavior Survey 


Yes 


Katsuyama and 
Kimble 2002 


Student report 


X 




Study sample, 
grades K-6 


0 


CL 


Student School Climate 
Questionnaire 


Yes 


Katsuyama and 
Kimble 2002 


Student report 




X 


Study sample, 
grades K-6 


0 


CL 


Teacher School Climate 
Questionnaire 


Yes 


Katsuyama and 
Kimble 2002 


Teacher report 




X 


Study sample, 
grades K-6 


0 


CL 


Perceived Value of 
Character Education 


Yes 


Katsuyama and 
Kimble 2002 


Student report 


X 




Study sample, 
grades K-6 


0 


CC! 


Attitudes Toward Ethical 
Issues — World View 
Subscale 


Yes 


Fruechte and 
Mitchell 2003 


Student report 


X 




Study sample, 
grades 7-12 


0 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 


Domain Assessed* 




Characteristics 


Reliability 

Rating 11 


Program 11 


Instrument Name b 


Citation/s d 


Affect Behav 


Cognit 


Other 


of Development 
Sample 8 


CC! 


Recent Behaviors — 
Behavior Toward Rules, 
Behavior Toward Others 
and Positive Behaviors 
subscales 


Yes 


Fruechte and 
Mitchell 2003 
Lennox School 
District 2007 


Student report 


X 






Study sample, 
grades 6-12 


0 


CC! 


Recent Behaviors — 
Parental Behaviors 
Subscale 


Yes 


Lennox School 
District 2007 


Student report 






X 


Study sample, 
grades 6-8 


0 


CC! 


Stanford Achievement 
Test, Ninth Edition 
(SAT 9) 


No 


Pearson 
Education, Inc. 
1996 


Direct assessment 




X 




Nationally 
representative, 
grades K-13 


0 


CC! 


California Achievement 
Test, Sixth Edition 
(CAT/6)/TerraNova, The 
Second Edition 


No 


CTB/McGraw- 
Hill n.d. (b) 


Direct assessment 




X 




Nationally 
representative, 
grades 3 and 7 


0 


Cp 


School Climate and 
Interactions Survey 


Yes 


Marshall and 
Caldwell 2007 


Student report 
Staff report 
Parent report 






X 


Study sample, 
grades 4, 8, and 
11 


1-2 


CQ 


Character Development 
Survey 


No 


Johns 1997 


Student report 
Parent report 
Teacher report 






X 


External 
sample, grades 
NR 


2 


C of C 


Character Development 
Survey 


No 


Johns 1997 


Student report 
Parent report 
Staff report 






X 


External 
sample, grades 
NR 


2 


C of C 


Upholding Values Survey 


Yes 


Voelker 1994 


Student report 
Staff report 






X 


Study sample, 
grades 7-9 


0 




TABLE 3 ( continued ) 







Developed 
for Study 6 




Type of 
Assessment 6 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


CWK 


Students’ Own Behavior 


Yes 


Page and 
D’Agostino 2005 


Student report 




X 




Study sample, 
grades 3, 4, 6- 
12 


0 


CWK 


Perceptions of Students’ 
Behavior 


Yes 


Page and 
D’Agostino 2005 


Student report 
Teacher report 








X Study sample, 

grades 3, 4, 6- 
12 


0 


COOL 

Kids 


California Standards Test 


No 


California 
Department of 
Education n.d. 


Direct assessment 






X 


External 
sample, grades 
2-12 


0 


E for C 


Survey of Perceptions of 
Oneself and School 


Yes 


Pinhas and Kim 
2004 


Student report 


X 


X 




Study sample, 
grades 7-8 


1-2 


GH 


(no scale reported) 


















HECC 


Ethical Understanding 
Scale 


Yes 


Leming, 
Henricks-Smith, 
and Antis 2000 


Student direct 

assessment 






X 


Study sample, 
grades 1-6 


1-2 


HECC 


Ethical Sensibility Scale 


Yes 


Leming, 
Henricks-Smith, 
and Antis 2000 


Student report 


X 






Study sample, 
grades 1-6 


2 


HECC 


Social Distance Task 


Adapted 


Koslin, Amarel, 
and Ames 1969 


Direct assessment 


X 






External 
sample, grades 
K-6 


2 


HECC 


Ethical Behavior Rating 


Yes 


Leming, 
Henricks-Smith, 
and Antis 2000 


Teacher report 




X 




Study sample, 
grades 1-6 


2 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 


Domain Assessed* 




Characteristics 


Reliability 

Rating 11 


Program 11 


Instrument Name b 


Citation/s d 


Affect Behav Cognit 


Other 


of Development 
Sample 8 


ICPS 


Latent School Climate 
Scale 

Behaviors Assessment 
for Children 
Parent Report on 
School Climate 

Parent and Teacher 

Involvement 

Questionnaire 


Yes 


Kumpfer, 
Alvarado, Tait 
and Turner 2002 

Reynolds and 
Kamphaus 1992 

Conduct 
Problems 
Prevention 
Research Group 
1999 


Student report 
Parent report 
Teacher report 




X 


Study sample, 
grade 1 (and 
parents, 
teachers) 


1-2* 


ICPS 


Parenting Skills Scale 
Parenting Practices 
Scale 


Yes 


Kumpfer, 
Alvarado, Tait 
and Turner 2002 


Parent report 




X 


Study sample, 
parents (of 
grade 1) 


1-2* 




Alabama Parenting 
Questionnaire 




Loeber, 
Farrington, 
Strouthamer- 
Loeber, and Van 
Kammen 1998 


















Shelton, Frick, 
and Wootton 
1996 












ICPS 


Behaviors Assessment 
for Children — 
Adaptability, Leadership, 
and Social Skills 
Subscales 


Adapted 


Reynolds and 
Kamphaus 1992 


Parent report 
Teacher report 


X 




Nationally 

representative, 

K-PSE 


2* 


ICPS 


Family Relations Scale 


Adapted 


Gorman- Smith, 
Tolan, Zelli, and 
Huesmann 1996 


Parent report 




X 


Study sample, 
parents (of 
grade 1) 


1-2* 




TABLE 3 ( continued ) 



to 



Developed 

Program 11 Instrument Name b for Study c Citation/s d 

ICPS Self Regulation Scale Yes Kumpfer, 

Parent Observation of Alvarado, Tait 

Classroom Adaptation- ant ^ Turner 2002 

Revised— Impulsivity, Werthamer- 

Hyperactivity, and Larsson, Kellam, 

Aggressive/Disruptive and Ovesen- 

Behavior subscales McGregor 1991 

Teacher Observation 
of Classroom 
Adaptation-Revised — 

Impulsivity, 

Hyperactivity, and 
Aggressive/Disruptive 
Behavior subscales 

ICE Milson’s Character No Milson2001 

Education Efficacy Belief 
Instrument 



ICE 


Respect for Self and 
Others 


Yes 


Cl Associates 
2006 


ICE 


Integrity and Respect 
(Think, Do and How 
Many Times) 

Character Education 
Survey — Respect and 
Integrity Subscales 


Adapted 


Cl Associates 
2006 

ETR Associates 
2000 


ICE 


Responsibility 


Yes 


Cl Associates 
2006 


JC 


Questionnaire on 
Engagement of Risky 
Behavior 


Yes 


Kuther and 
Higgins- 
D ’Alessandro 
2000 




Teacher report 






X External 

sample, grades 
NR 


1-2* 


Teacher report 


X 


X 


Study sample, 
grades 4, 7-9 


0 


Student report 


X 


X 


Study sample, 
grade 4 / 
External 
sample, grades 
NR 


0 


Teacher report 




X 


Study sample, 
grades 4, 7-9 


0 


Student report 


X 


X 


Study sample, 
grades 10-12 


0 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


JC 


Perceptions of Risky 
Behavior 


Yes 


Kuther and 
Higgins- 
D’Alessandro 
2000 


Student report 


X 




X 


Study sample, 
grades 10-12 


0 


JC 


Defining Issues Test 


Adapted 


Rest 1986, 1994 


Direct assessment 






X 


External 
sample, grades 
NR 


2* 


JC 


Standard Moral Judgment 
Interview 


Adapted 


Colby et al. 1987 


Indirect assessment 






X 


External 
sample, grades 
NR 


0 


LFL 


(no scale reported) 


















LIFT 


Walker-McConnell Scale 
of Social Competence 
and School Adjustment — 
Peer Preferred Social 
Behavior Subscale 


No 


Walker and 
McConnell 1995 


Teacher report 




X 




External 
sample, grades 
NR 


2 


LQ Skills 
for 

Action 


Life Review Survey 


Yes 


Laird, Bradley, 
and Black 1998 


Student report 


X 


X 




Study sample, 
grades 9-12 


0 


LQ Skills 
for 

Action 


Service-Learning Survey 


Adapted 


Blyth, Satino, 
and Berkas 1997 


Student report 


X 


X 




External 
sample, grades 
NR 


0 


LQ Skills 
for 

Action 


Checklist of Personal 
Gains 


Adapted 


Conrad and 
Hedin 1980 


Student report 


X 






External 
sample, grades 
NR 


0 


LQ Skills 
for Adol 


Perception of Substance 
Use Survey 

Monitoring the Future 
Survey 


Yes 


Eisen, Zellman, 
and Murray 2003 

Johnston, 
O’Malley, and 
Bachman 1996 


Student report 


X 






Study sample 
, grades 6-9 


2 




TABLE 3 ( continued ) 







Developed 
for Study 6 




Type of 
Assessment 6 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


LQ Skills 
for Adol 


Parental Monitoring 


Yes 


Eisen, Zellman, 
and Murray 2003 


Student report 








X Study sample, 

grades 6-9 


1 


OC 


Social Skills Rating 
System — Social Skills, 
Problem Behaviors and 
Academic Competence 
Scales 


No 


Gresham and 
Elliott 1990 


Student report 
Parent report 
Teacher report 


X 


X 


X 


Nationally 
representative, 
grades 3-12 


1-2* 


OC 


Relational Health 
Indices — Youth Version 


No 


Liang, Tracy, 
Kenny, Brogan, 
and Gatha (under 
review) 


Indirect assessment 


X 






External 
sample, grades 
NR 


2* 


OC 


Survey of Adaptation 
Tasks — Middle School 


Adapted 


Elias et. al 1992 


Student report 




X 




External 
sample, grades 
NR 


2 


OC 


Parent Report of School 
Adjustment 


Yes 


Taylor, Liang, 
Tracy, Williams 
and Siegle 2002 


Parent report 


X 






Study sample, 
grade 6 


2 


PCE 


Pennsylvania System of 
School Assessment 


No 


Pennsylvania 
Department of 
Education 


Direct assessment 






X 


NR 


0 


PCE 


TerraNova Sup era 


No 


CTB/McGraw- 
Hill n.d. (c) 


Direct assessment 






X 


Nationally 
representative 
of Spanish- 
speaking 
students, grades 
1-10 


2*t 


PCE 


Survey of Student 
Behavior and Affect 


Yes 


RMC Research 
Corporation 2007 


Student report 


X 


X 




Study sample, 
grades 6-12 


1-2* 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 




Domain Assessed* 




Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


Other 


of Development 
Sample 8 


PA 


Student Behavior 
Checklist 

Child Behavior Profile 
Teacher Child Rating 
Scale 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 
Edelbrock and 
Achenbach 1984 
Hightower et al. 
1986 


Teacher report 




X 






Study sample, 
grades 1-5 


1-2 


PA 


Involvement in and 
Interpretation of 
Behaviors 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Student report 


X 


X 






Study sample, 
grades 1-5 


1-2 


PA 


Feelings Toward Others 
and the School 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Student report 


X 








Study sample, 
grades 1-5 


1-2 


PA 


Attitudes and Intentions 
Regarding Positive 
Action 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Student report 


X 








Study sample, 
grades 1-5 


2 


PA 


Survey of Teacher 
Attitudes 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Teacher report 








X 


Study sample, 
grades NR 


2 


PA 


Experiences of 
Harassment at School 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Student report 








X 


Study sample, 
grades 1-5 


2 


PA 


Family Participation in 
School 


Yes 


Flay, Acock, 
Vuchinich, and 
Beets 2006 


Student report 








X 


Study sample, 
grades 1-5 


1 


PA 


Hawaii Content and 
Performance Standards 
Test 


No 


Accountability 
Resource Center 
Hawaii, Hawaii 
Department of 
Education n.d. 

(a) 


Direct assessment 






X 




External 
sample, grades 
NR 


0 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav Cognit 


of Development 
Other Sample 8 


PA 


Hawaii Department of 
Education School Quality 
Survey — Parent 
Involvement Subscale 


No 


Accountability 
Resource Center 
Hawaii, Hawaii 
Department of 
Education n.d. 

(b) 


Parent report 






X External 

sample, grades 
NR 


0 


PE 


Behavioral Inventory 


Yes 


Dunn and Wilson 
n.d. 


Teacher report 




X 


Study sample, 
grades PreK-8 


2 


H3 


Character Asset 
Questionnaire 


Yes 


Furco et al. 2004 


Student report 


X 


X 


Study sample, 
grades 2-6 


1-2 


RHC 


Commitment to School 


No 


Hawkins et 
al. 1 992 


Parent report 
Teacher report 


X 




External 
sample, grades 
NR 


l-2 f 


RHC 


Academic Performance 
Rating 


Yes 


Catalano et al. 
2003 


Parent report 
Teacher report 




X 


Study sample, 
grades 1-2 


1-2* 


RHC 


Interpersonal 

Competency 

Teacher Observation 
of Classroom 
Adaptation — Revised 

Walker-McConnell 
Scale of Social 
Competence 


Yes 


Harachi et al. 
1999 

Werthamer- 
Larsson, Kellam, 
and Ovesen- 
McGregor 1990 
Walker and 
McConnell 1988 


Student report 
Parent report 
Teacher report 


X 




Study sample, 
grades 1-2 


2 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav Cognit 


of Development 
Other Sample 8 


RHC 


Antisocial Behavior 
Teacher Observation 
of Classroom 
Adaptation — Revised 
Teacher Report 
Form — Aggressive 
Behavior Subscale 


Yes 


Catalano et al. 
2003 

Wethamer- 
Larsson, Kellam, 
and Ovesen- 
McGregor 1990 

Achenbach 1991 

Achenbach and 
Edelbrock 1983 


Student report 
Parent report 
Teacher report 


X 




Study sample, 
grades 1-2 


2 


RCCP 


Teacher Checklist of 
Child Aggressive 
Behavior 


Adapted 


Dodge and Coie 
1987 


Teacher report 




X 


External 
sample, grades 
3-6 


2 


RCCP 


Social Competence 
Scale — Prosocial 
Communication and 
Emotional Regulation 
Subscales 


Adapted 


Conduct 
Problems 
Prevention 
Research Group 
1991 


Teacher report 




X 


External 
sample, grades 
K-3 


2 


RCCP 


Seattle Personality 
Inventory — Depression 
And Conduct Problems 
Subscales 


Adapted 


Greenberg 1994 


Student report 


X 




External 
sample, grades 
1-5 


1-2 


RCCP 


Home Interview — Hostile 
Attribution and 
Aggressive Negotiation 
Subscales 


Adapted 


Dodge 1986 


Indirect assessment 


X 




External 
sample, grades 
K-3 


1-2 


RCCP 


Social Problem Solving 
Measure 


No 


Lochman and 
Dodge 1994 


Indirect assessment 




X 


External 
sample, grades 
NR 


1 




TABLE 3 ( continued ) 







Developed 
for Study 6 




Type of 
Assessment 6 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


RCCP 


What I Think 


No 


Rosenfeld, 
Huesmann, Eron, 
and Torney-Pura 
1982 


Student report 


X 






External 
sample, grades 
NR 


1 


SCPP- 

YA 


Coping Skills 

Decision-Making 

Questionnaire 


Yes 


Caplan et al. 
1992 

Gersick, Grady, 
and Snow 1988 


Indirect assessment 


X 


X 


X 


External 
sample, grades 
NR 


2 f 


SCPP- 

YA 


Rand Well-Being Scale 


No 


Veit and Ware 
1983 


Student report 


X 






Nationally 

representative 

sample 


2 


SCPP- 

YA 


Self-Perception Profile 
for Children — Behavioral 
Conduct and Self-Worth 
Subscales 


No 


Harter 1985 


Student report 


X 






External 
sample, grades 
NR 


2 


SCPP- 

YA 


Decision-Making 
Confidence Scale 


Adapted 


Wills 1986 


Student report 


X 






External 
sample, grades 
NR 


1 


SCPP- 

YA 


Problem-Solving 

Efficacy 


No 


Weissberg, 
Barton, and 
Shriver 1997 


Student report 


X 






External 
sample, grades 
NR 


2 


SCPP- 

YA 


Attitudes Towards 
Substance Use 


No 


Botvin, Baker, 
Renick, 
Filazzola, and 
Botvin 1984 


Student report 


X 






External 
sample, grades 
NR 


2 


SDM/PS 


Youth Self Report 


Adapted 


Achenbach and 
Edelbrock 1987 


Student report 




X 




External 
sample, grades 


1-2* 



6-12 




TABLE 3 ( continued ) 







Developed 
for Study 0 




Type of 
Assessment 0 




Domain Assessed* 


Characteristics 


Reliability 

Rating* 1 


Program 11 


Instrument Name b 


Citation/s d 


Affect 


Behav 


Cognit 


of Development 
Other Sample 8 


SDM/PS 


Perceived Competence 
Scale for Children 


No 


Flarter 1982 


Student report 


X 






External 
sample, grades 
3-9 


1-2* 


SDM/PS 


Comprehensive Test of 
Basic Skills 


No 


CTB/McGraw- 
Hill n.d. (b) 


Direct assessment 






X 


Nationally 

representative 


0 


SDM/PS 


Group Social Problem 
Solving Assessment 


Adapted 


Elias, Rothbaum, 
and Gara 1986 


Indirect assessment 






X 


External 
sample, grades 
3-4 


2* 


TO 


(no scale reported) 


















TGFDV 


Intentions Scale 


Yes 


Bacon 2001 


Student report 


X 






Study sample, 
grades 9-12 


2 


TGFDV 


Protective Factor 
Perceptions Survey 


Yes 


Bacon 2001 


Student report 


X 






Study sample, 
grades 9-12 


1-2 


TGFV 


Student Protective Factor 
Survey 


No 


Bacon 2000; 
2001 


Student report 


X 


X 


X 


External 
sample, grades 
6-12 


2 


TGFV 


Teacher Checklist of 
Student Behaviors 


No 


Bacon 2003 


Teacher report 




X 


X 


External 
sample, grades 
K-6 


2 


Topeka 
Char. 1st 


Topeka Character 
Education Survey 


Yes 


Tatarko 2007 


Student, teacher, 
and parent report 


X 


X 




X Study sample, 

grades 4, 7, 9 


0 


Tribes 

TLC 


(no scale reported) 


















V-LACE 


Group for the Study of 

Interpersonal 

Development 

Relationship 

Questionnaire 


No 


Schultz, Selman, 
and LaRusso 
2003 


Student report 




X 


X 


External sample, 
grades K-8 and 
12 


1-2* 




TABLE 3 ( continued ) 



“In this table, program names have generally been abbreviated. Table 1 includes complete program names alongside the abbreviations. 

b ln instances where the instrument was developed by researchers for the study reviewed here, we use as the instrument name the researchers’ own descriptive 
terms regarding the content. Where an published instrument was used, we report that name, even when the instrument was adapted for the purposes of the study 
(indicating adaptation in the next column to the right). Indented items in the “Instrument Name” column reflect use by the researchers of selected items from 
that instrument in combination with other items (either from other published measures or developed by the researchers themselves). 

c If an instrument was not developed for the study reviewed here and no adaptations were made to that instrument, we report “No” in this column. If an 
instrument was developed for the study or selected items from one instrument were combined with items from another instrument to form a single scale, we 
report “Yes.” If an existing instrument was simply shortened or researchers rephrased particular items but left the measure largely intact, we report “Adapted.” 

d Full citations for all instruments are presented in the References at the end of this report. 

c In this column we report both the type of assessment and respondent/report. Where measures/instruments were completed by more than one reporter, they are 
all listed. 

* The “Domain Assessed” column captures the broad categories in which the constructs measured by a particular instrument fall. “Affect,” “Behav,” and 
“Cognit” all refer to student-level outcomes. “Other” refers to outcomes at the school, teacher/administrator, or parent/community level. A single instalment 
can reflect multiple domains (i.e., checkmarks in multiple domain columns for a single instrument). 

8 This column summarizes characteristics of the sample used to develop the instrument. Sample types are categorized as either the “study sample” (i.e., the 
measure was developed on the sample studied in the program report); an “external sample” (i.e., the measure was developed using a non-nationally 
representative sample for research other than the study reviewed here); or “national sample” (i.e., the measure was developed with an external, nationally 
representative sample). 

h For its reliability rating, an instrument can receive a “0” (insufficient information provided to gauge the instrument’s reliability), “1” (reported reliability less 
than .70), or “2” (reported to have a reliability (any type) equal to or exceeding .70). Where a range is given (e.g., 1-2), this indicates that different levels of 
reliability were reported across subscales of a single instrument, among different subpopulations, or among different reporters (e.g., students, parents, teachers). 
Unless otherwise indicated (see notes below), these values refer to internal consistency reliability. 

^Reliability reported for development sample, rather than study sample. 

NR=Not Reported. 

tAlternative measures of reliability were reported for these instruments including the following: BDS/Janis-Field Feeling of Inadequacy Scale — split-half 
reliability; PCE/TerraNova Supera — Kuder Richardson 20 coefficients; RFIC/Commitment to School — correlation; RFIC/Academic Performance Rating, parent 
measure — correlation; SCPP-YA — inter-rater reliability. 




A number of findings emerge from the information presented in Table 3. These include 
findings on the use of new and existing measures, measure reliability, and measure validity, each 
of which is discussed below. Supporting evidence provided in program-specific tables in the 
appendix augments these findings. The appendix material points to issues regarding single-item 
measurement, which is discussed in the final section of this chapter. 



Use of New vs. Existing Measures. Across the studies reviewed here on the 36 programs, 
researchers used 95 scale measures. In the discussion that follows, these 95 scales are the unit of 
analysis. About half of these (46) were developed specifically for the study under review. Of the 
remaining 49 scales, 17 were adapted from existing measures and 32 were developed by other 
researchers and/or for other studies (and were available “off the shelf’ for the authors of these 
studies to use). Forty-four of the 46 newly developed scale measures were surveys of students, 
teachers, or parents. These new measures were developed to gauge constructs in the affective (20 
instances), behavioral (20), or other domains (15), with relatively few new scales measuring 
cognitive constructs (5); instead, researchers relied on existing measures, largely published 
standardized tests, to assess cognitive outcomes (15). 

Among existing published measures, six were employed in research on more than one 
program in our group. The Social Skills Rating System (SSRS; Gresham and Elliot 1990), the 
Teacher Observation of Classroom Adaptation-Revised (TOCA-R; Werthamer-Larsson et al. 
1991), the Walker-McConnell Scale of Social Competence (Walker and McConnell 1995), the 

o 

Character Development Survey (Johns 1997), the California Standards Tests (California 
Department of Education), and the California Achievement Test, Sixth Edition/TerraNova, The 
Second Edition (CTB/McGraw Hill) were each used in studies of two programs. Most of these 
published scales were used as designed, but researchers selected items from both the TOCA-R 
and the Walker-McConnell to create new scales in studies of the Raising Healthy Children 
(RHC) program. 

Inconsistent application of the same or demonstrably comparable measures across studies 
poses a potential challenge to the field of character education: Although conceptually similar 
outcome constructs are assessed in the research reviewed here, outcomes are rarely measured 
with the same instruments. This complicates comparison of outcomes across studies of one or 
more programs. Adding to the challenge, relatively little information on measurement properties 
is reported, as described in the next section. 



Reliability of Measures. To assess the psychometric properties of measures used in studies 
of character education programs, we focus primarily on information regarding measure 
reliability. For about two-thirds of the scaled measures employed (62 of 95 scales), information 
on reliability was available either through the study under review or through a cited source. In 57 
of these cases, a measure of internal consistency reliability, typically Cronbach’s alpha, was 



8 According to Laing et al (1999), the Character Development Survey was developed by Johns (1997) for their 
evaluation of the Utah Community Partnership for Character Education Development. The evaluation of the 
Character Quality Program by the Chugach School District (2004) reported retrieving this instrument through the 
Character Education Partnership “Evaluation Toolkit” at http://www.character.org. 
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presented (alternative reliability measures are discussed in the notes to Table 3). The focus on 
internal consistency reliability indicates that researchers aimed to measure single, cohesive 
constructs with each scale or subscale. In the one instance where researchers employed a 
measure other than internal consistency reliability — a measure of Coping Skills used in a study 
of the Social Competence Promotion Program for Young Adolescents — inter-rater reliability was 
assessed. Among the 62 scale measures with reliability information, 30 have reliability of.70 or 
above (indicated by a “2” in the reliability column of Table 3); 27 reported reliability that varied 
across subscales, subpopulations, or reporters; and 5 reported reliability of measures below .70. 9 

For about one-third (33 of 95) of the scale measures presented in Table 3, information 
provided in the studies reviewed or in references provided in the studies was insufficient to 
assess the reliability of the measure in any form (indicated by a “0” in the reliability column). 
Among instruments that were developed for the study reviewed, no reliability information was 
available for 19 of 46 cases. Among the scale measures for which no information on reliability 
was presented, it is possible that instrument developers did assess reliability, but did not report 
the infonnation in any of the studies reviewed here. 

Table 4 lists, for each category in our taxonomy, measures with acceptable levels of reported 
reliability. We have included all measures from studies of the character education programs 
examined in this research that have reported reliability of .70 or better. For constructs with no 
measure exceeding .70 reliability but with measures above .60 reliability, we have also included 
the measure with the highest reliability. A number of measures are cited as capturing more than 
one category in our taxonomy. Brief descriptions of each scale and references to the literature are 
presented in the Appendix A entry for the program whose acronym is shown in the second 
column of Table 4. Readers should note that these are not necessarily the “best” measures 
available for assessing a given construct. Rather, they are examples of scaled measures with 
demonstrated reliability that might be appropriate for a given construct. 



The table also shows those constructs for which the reviewed studies lacked a measure 
meeting our standard for inclusion in Table 4. Three constructs were not measured using a scale 
(indicated by “NM” in the second column): child knowledge of risk prevention, child healthy 
lifestyle, and teacher knowledge of child development. In addition, 1 1 constructs were measured 
with a scale in at least one study, but none of the studies reported the reliability of the scale 
(indicated by “NR” in the second column of Table 4). Some of these constructs — for example, 
absence/tardiness and crime — may not lend themselves to measurement using scales. But others 
are potentially important, multifaceted character outcomes. In particular, justice/fairness and 
trustworthiness are not measured with demonstrated reliability — despite the fact that both 
constructs are among the “six pillars” of character identified by the Josephson Institute 2009 and 
the first of these two is among the 24 positive character traits of the VIA classification (Peterson 
and Seligman 2004). Still, in reviewing Table 4, it is important to keep in mind, first, that these 
constructs may have been measured with single items and/or non-scaled measures (see Appendix 
A) and, second, that for 33 of 95 scales, reliability was not reported. 



9 See Sattler (2001) for a discussion of the designation of .7as a cut-point for relatively high reliability. 
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TABLE 4 



OVERVIEW OF MEASURES WITH HIGHEST REPORTED RELIABILITY FOR CHARACTER EDUCATION OUTCOME CONSTRUCTS 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


Student-Level Outcomes 








Cognitive 








Knowledge 








understanding 

values/norms 


Ethical Understanding Scale (HECC) 


.70 for grade 4-6 version 


Lemming, Henricks-Smith, and 
Antis 2000 


risk prevention 


NM 






interpersonal knowledge 


Student Protective Factors Survey (TGFV) 


.83 for Social and Conflict Resolution 
and Resistance Skills subscale; .82 
Communication Skills subscale 


Bacon 2000; 2001 




Group for the Study of Interpersonal 
Development Relationship Questionnaire 
(V-LACE) 


GSID-Rel Q: .72-. 87 depending subscale 


Schultz, Selman, and LaRusso 
2003 


intrapersonal knowledge 


Student Protective Factors Survey (TGFV) 


SPFS: .80 Emotional Competency 
subscale 


Bacon 2001; 2001 




Teacher Checklist of Student Behaviors 
(TGFV) 


TCSB: .91 Personal and Social skills 
subscale 


Bacon 2003 


academic content 


Social Skills Rating System Academic 
Competence Scale (OC) 


SSRS-ACS: .95 


Gresham and Elliott 1990 




TerraNova Supera (PCE) 


TNS: .80-. 90 depending on subtest 


CTB/McGraw-Hill n.d. (c) 




Academic Performance Rating (RHC) 


APR: .92 


Catalano et al. 2003 


Reasoning 








moral/ethical reasoning 


Defining Issues Test (JC) 


DIT: averages in the .80s for different 
age groups 


Rest 1986, 1994 




Home Interview (RCCP) 


HI: .74-. 78 for Hostile Attribution 
subscale; .87-. 90 for Aggressive 
Negotiation subscale 


Dodge 1986 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 




Coping Skills-Decision Making 
Questionnaire (SCPP-YA) 


CSDMQ: .81-. 93 


Caplan et al. 1992; Gersick, Grady, 
and Snow 1988 


critical thinking/ decision 
making 


Group Social Problem Solving Assessment 
(SDM/PS) 


.75-. 85 cited from other studies 


Elias, Rothbaum, and Gara 1986 


Affective 








Attitudes/Motives 


Survey of Perceptions of Oneself and 
School (E for C) 


SPOS: .64-. 88 depending on subscale and 
pre/post 


Pinhas and Kim 2004 




Attitudes and Intentions Toward Positive 
Action (PA) 


AITPA: .91 for attitudes and .82 for 
intentions 


Flay, Acock, Vuchinich, and Beets 
2006** 




Involvement In and Interpretation of Own 
Behaviors (PA) 


IIIOB: .68-. 78 depending on wave 


Flay, Acock, Vuchinich, and Beets 
2006** 




Home Interview (RCCP) 


HI: .74-. 78 for Hostile Attribution 
subscale; .87-. 90 for Aggressive 
Negotiation subscale 


Dodge 1986 




Problem Solving Efficacy (SCPP-YA)P 


PSE: .71 


Weissberg, Barton, and Shriver 
1997 


prosocial dispositions 


Social Relatedness Scale (BDS) 


SRS: Sense of Social Responsibility 
(School), .70 pre and .76 post 
intervention 


Newmann and Rutter 1983 




Ethical Sensibility Scale (HECC) 


ESS: .80 and .83 for grades 1-3 and 4-6, 
respectively 


Leming, Henricks-Smith, and Antis 
2000 




Self-Perception Profile for Children (SCPP- 
YA) 


SPPC: .75 for behavioral conduct scale 
and .76 for self worth scale 


Harter 1985 




Protective Factor Perceptions Survey 
(TGFDV) 


PFPS: .93 overall, .59 to .85 for subscales 


Bacon 2001 


attitudes toward school 


Feelings Toward Others and the School 
(PA) 


FTOS: .78-.85 for grade 3 and up overall, 
.71-. 79 for subscales 


Flay, Acock, Vuchinich, and Beets 
2006** 




Survey of Student Behavior and Affect 
(PCE) 


SSBA: .82 for value of school subscale 


RMC Research Corporation 2007 




Commitment to School (RHC) 


CS: .80 for teacher reports 


Hawkins et al.1992 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


attitudes toward 
risk/health 


Perceptions of Substance Use Survey (LQ- 
Skills for Adol) 


PSUS: .73-. 89, depending on subscale 
and substance 


Eisen, Zellman, and Murray 2003 




Attitudes Toward Substance Use (SCPP- 
YA) 


ATSU: .74 for smoking attitudes; .80 for 
drinking attitudes 


Botvin, Baker, Renick, Filazzola, 
and Botvin 1984 




Intentions Scale (TGFDV) 


IS: .74 


Bacon 2001 




Protective Factor Perceptions Survey 
(TGFDV) 


PFPS: .93 overall, .59 to .85 for subscales 


Bacon 2001 


civic dispositions 


Social Relatedness Scale (BDS) 


.78-. 80 for Future Participation; .71-. 72 
for Future Affiliation 


Newmann and Rutter 1983 


attitudes toward diversity 


Social Distance Task (HECC) 


.80 for grades 1-3; .85 for grades 4-6 


Koslin, Amarel, and Ames 1969 


intrapersonal strengths 
(self-esteem, self- 
efficacy) 


Janis-Field Feeling of Inadequacy Scale 
(BDS) 


JFFIS: .72-. 88 


Robinson and Shaver 1973 




Seattle Personality Inventory (RCCP) 


SPI: .73-. 76 for Depression subscale 


Greenberg 1994 




Coping Skills-Decision Making 
Questionnaire (SCPP-YA) 


CS-DMQ: .81-. 93 


Caplan et al. 1992; Gersick, Grady, 
and Snow 1988 




Self-Perception Profile for Children (SCPP- 
YA) 


SPPFC: .75 for behavioral conduct scale; 
.76 for self-worth scale 


Harter 1985 




Perceived Competence Scale for Children 
(SDMPS) 


PCSC: .73-. 86 depending on subscale and 
age 


Harter 1982 


internalizing problems 


Seattle Personality Inventory (RCCP) 


SPI: .73-. 76 for Depression subscale 


Greenberg 1994 




Rand Well-Being Scale (SCPP-YA) 


RWBS: 82 


Veit and Ware 1983 




Youth Self Report (SDMPS) 


YSR: .71-. 95 


Achenbach and Edelbrock 1987 


Attitudes/Emotions 


Survey of Perceptions of Oneself and 
School (E for C) 


SPOS: .64-. 88 depending on subscale and 
pre/post 


Pinhas and Kim 2004 




Relational Health Indices- Youth Version 
(OC) 


RHI-YV: .80-. 86 depending on subscale 


Liang, Tracy, Kenny, Brogan, and 
Gatha (under review) 


caring (e.g., empathy) 


Social Skills Rating System Social Skills 
Scale (BEST, OC) 


.74 for student empathy subscale 


Gresham and Elliott 1990 


reflectivity 


NR 









Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


school bonding/school 
engagement 


Enhanced Relationships Survey (CSC) 


ERS: .78 for Liking for School subscale, 
.74 for Sense of Community subscale, .73 
for Trust in and Respect for Teachers 
subscale 


Developmental Studies Center n.d. 




Parent Report of School Adjustment (OC)' 


PRSA: .76 


Taylor, Liang, Tracy, Williams and 
Siegle 2002 




Feelings Toward Others and the School 
(PA) 


FTOS: .78-. 85 for grade 3 and up overall, 
.71-. 79 for subscales 


Flay, Acock, Vuchinich, and Beets 
2006** 


justice, fairness 


NR 






Behavioral 








Competencies/skills 


Survey of Perceptions of Oneself and 
School (E for C) 


SPOS: .64-. 88 depending on subscale and 
pre/post 


Pinhas and Kim 2004 




Student Behavior Checklist (PA) 


SBC: .54-. 91 depending on subscale (on 
3 of 17 subscales have alpha<.7) 


Flay, Acock, Vuchinich, and Beets 
2006** 


resistance 


Coping Skills-Decision Making 
Questionnaire (SCPP-YA) 


CS-DMQ: .81-. 93 


Caplan et al. 1992; Gersick, Grady, 
and Snow 1988 




Student Protective Factors Survey (TGFV) 


SPFS: .83 Social Conflict Resolution and 
Resistance Skills subscale 


Bacon 2000; 2001 


responsibility 


Social Skills Rating System (OC) 


SSRS: .84 for Responsibility subscale 


Gresham and Elliott 1990 




Behavioral Inventory (PE) 


BE. 94 




integrity 


Character Asset Questionnaire (H3) 


.61 Integrity and Honesty subscale 


Furco et al. 2004 


respect 


Behavioral Inventory (PE) 


BE .94 


Dunn and Wilson n.d. 




Teacher Checklist of Student Behaviors 
(TGFV) 


TCSB: ..91 for Personal and Social Skills 
subscale 


Bacon 2003 


leadership 


Behavioral Assessment System for 
Children (ICPS) 


.85 for Leadership Subscale 


Reynolds and Kamphaus 1992 


intrapersonal competency 
(self-control, self- 
discipline, self- 
regulation) 


Social Skills Rating System (BEST, OC) 


SSRS: .94 for teacher and .83 for student 
for total scale, .92 for teacher for 
Cooperation subscale; .91 for teacher for 
Self-Control subscale; .88 for Problem 
Behaviors subscale 


Gresham and Elliot 1990 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 




Self-Regulation Scale (ICPS) 


SRS: .85-. 96 for subscales drawn from 
Teacher Observation of Classroom 
Adaptation-Revised 


Werthamer-Larsson, Kellam, and 
Ovesen-McGregor 1991 




Behavioral Inventory (PE) 


Bl: .94 


Dunn and Wilson n.d. 




Student Protective Factors Survey (TGFV) 


SPFS: .80 Emotional Competency 
subscale 


Bacon 2000; 2001 


interpersonal competency 


Social Skills Rating System (BEST, OC) 


SSRS: .94 for teacher and .83 for student 
for total scale, .74 for student for 
Empathy subscale, .75 for teacher for 
Assertion subscale; .88 for Problem 
Behaviors subscale 


Gresham and Elliot 1990 




Behavioral Assessment System for 
Children (ICPS) 


BASC: .89 for Social Skills subscale, .74 
for Adaptability subscale 


Reynolds and Kamphaus 1992 




Walker-McConnell Scale of Social 
Competence and School Adjustment (LIFT) 


WMS: .94 pre- and .96 post-intervention 


Walker and McConnell 1995 




Survey of Adaptation Tasks-Middle School 
(OC) 


SAT-MS: .86 reduced scale, .92 full scale 


Elias et. al 1992 




Behavioral Inventory (PE) 


Bl: .94 


Dunn and Wilson n.d. 




Interpersonal Competency-teacher measure 
(RHC) 


IC-tm: .94 


Harachi et al. 1999; Werthamer- 
Larson, Kellam, and Ovesen- 
McGregor 1990; Walker and 
McConnell 1988 




Interpersonal Competency-parent measure 
(RHC) 


IC-pm: .79 


Harachi et al. 1999 




Youth Self Report (SDMPS) 


YSP: .71-. 95 


Achenbach and Edelbrock 1987 




Student Protective Factor Survey (TGFV) 


SPFS: .83 for Social and Conflict 
Resolution and Resistance Skills 
subscale, .82 for Communication Skills 
subscale 


Bacon 2000; 2001 




Teacher Checklist of Student Behaviors 
(TGFV) 


TCSB: .93 Positive Social Behaviors 
subscale 


Bacon 2003 


communication 


Student Protective Factors Survey (TGFV) 


.82 for Communication Skills subscale 


Bacon 2000; 2001 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


coping 


Coping Skills-Decision Making 
Questionnaire (SCPP-YA) 


.81-.93 


Caplan et al. 1992; Gersick, Grady, 
and Snow 1988 


Prosocial behaviors 


Survey of Perceptions of Oneself and 
School (E for C) 


SPOS: .64-. 88 depending on subscale and 
pre/post 


Pinhas and Kim 2004 




Student Behavior Checklist (PA) 


SBC: .54-. 91 depending on subscale (3 of 
17 subscales have alpha<.7) 


Flay, Acock, Vuchinich, and Beets 
2006**; Edelbrock and Achenbach 
1984; Hightower et al. 1986 




Involvement In and Interpretation of Own 
Behaviors (PA) 


IIIOB: .68-. 78 depending on wave 


Flay, Acock, Vuchinich, and Beets 
2006** 


service 


Survey of Student Behavior and Affect 
(PCE) 


SSBA: .77 altruism subscale 


RMC Research Corporation 2007 




Social Competence Scale (RCCP) 


SCS: .98 


Conduct Problems Prevention 
Research Group 1991 




Character Asset Questionnaire (H3) 


CAQ: .76 caring and social equity 
subscale 


Furco et al. 2004 


healthy lifestyle 


NM 






kindness 


Survey of Student Behavior and Affect 
(PCE) 


SSBA: .77 altruism subscale 


RMC Research Corporation 2007 




Character Asset Questionnaire (H3) 


CAQ: .76 caring and social equity 
subscale 


Furco et al. 2004 


trustworthiness 


NR 






justice, fairness 


NR 






positive participation 


Social Relatedness Questionnaire (BDS) 


SRS1: Sense of Social Responsibility 
(School) subscale, .70 pre and .76 post 
intervention 


Newman and Rutter 1983 


Risk behaviors 


Involvement In and Interpretation of Own 
Behaviors (PA) 


IIIOB: .68-. 78 depending on wave 


Flay, Acock, Vuchinich, and Beets 
2006** 




Antisocial Behavior (RHC) 


AB: .78 for parents, .92 for teachers 


Catalano et al. 2003; Wethamer- 
Larsson, Kellam, and Ovesen- 
McGregor 1990; Achenbach 1991; 
Achenbach and Edelbrock 1983 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 




Youth Self Report (SDMPS) 


YSR: .71-. 95 


Achenbach and Edelbrock 1987 




Student Protective Factor Survey (TGFV) 


SPFS: .79 for Interactions with Others 
subscale 


Bacon 2000; 2001 




Teacher Checklist of Student Behaviors 
(TGFV) 


TCSB: .94 inappropriate behaviors 
subscale 


Bacon 2003 


substance use 


Perceptions of Substance Use Survey (LQ 
Skills for Adol) 


PSUS: .73 to .83 


Eisen, Zellman, and Murray 2003 




Survey of Adaptation Tasks-Middle School 
(OC) 


SAT-MS: .93 


Elias et. al 1992 


sexual risk-taking 


NR 






Violence 


Teacher Checklist of Child Aggressive 
Behavior (RCCP) 


TCCAB: .95 


Dodge and Coie 1987 




Seattle Personality Inventory (RCCP) 


SPI: ..73-. 79 for Conduct Problems 
subscale 


Greenberg 1994 


absence/tardiness 


NR 






discipline issues 


Survey of Adaptation Tasks-Middle School 
(OC) 


SAT-MS: .85 


Elias et. al 1992 




Teacher Checklist of Child Aggressive 
Behavior (RCCP) 


TCCAB: .95 


Dodge and Coie 1987 




Seattle Personality Inventory (RCCP) 


SPI: ..73-. 79 for Conduct Problems 
subscale 


Greenberg 1994 


crime 


NR 






T eacher/Admin-Le vel 
Outcomes 








Knowledge of child 
development 


NM 






Support (endorse or value what 
goes on in school) 


Survey of Teacher Attitudes (PA) 


.71-. 97 depending on subscale 


Flay, Acock, Vuchinich, and Beets 
2006** 


Understand values/norms 


Survey of Teacher Attitudes (PA) 


.71-.97 depending on subscale 


Flay, Acock, Vuchinich, and Beets 
2006** 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


Attendance 


NR 






Staff morale 


Enhanced Relationships Survey (CSC) 


ERS: .64 for Enjoyment of Teaching 
subscale 


Developmental Studies Center n.d. 




Survey of Teacher Attitudes (PA) 


STA: .71-. 97 depending on subscale 




Professional efficacy 


Milson’s Character Education Efficacy 
Belief Instrument (ICE) 


MCEEB1: .79 for Personal Teacher 
Efficacy subscale 


Milson 2001 




Survey of Teacher Attitudes (PA) 


STA: .93 for Teaching Self-Concept 
subscale, .94 for Self-Efficacy to Teach 
Positive Behaviors Generally subscale, 
.97 for Self-Efficacy to Teach Specific 
Positive Behaviors subscale 


Flay, Acock, Vuchinich, and Beets 
2006** 


School-Level Outcomes 








School climate 








collective norms/values 


School Climate and Interaction Survey 
(Cp); 


SCIS: .89 for students and .95 for staff 
and parents for School Expectations 
subscale 


Marshall and Caldwell 2007 




Character Development Survey (CQ; C of 
C) 


CDS:.70-.92 depending on subscale and 
reporter 


Johns 1997 


interactions among 
students, staff, parents 
(positive/ negative 
content) 


Enhanced Relationships Survey (CSC) 


ERS: .88 for Positive Relationships 
Among Students subscale, .84 for Faculty 
Collegiality subscale, .90 for Principal 
Accessibility and Supportiveness 
subscale 


Developmental Studies Center n.d. 




School Climate and Interaction Survey (Cp) 


SCIS: .92-,93 for Feelings of Belonging 
subscale depending on reporter 


Marshall and Caldwell 2007 




Character Development Survey (CQ; C of 
C) 


CDS : .91 parents, .89 staff for Parent- 
Staff Relationships subscale 


Johns 1997 




Latent School Climate (ICPS) 


LSC: >.7 for items from Behavioral 
Assessment System for Children 
depending on wave and/or source of 
alpha (study VS norming sample); >.7 for 
items from Parent Teacher Involvement 
Questionnaire 


Kumpfer, Alvarado, Tait and 
Turner 2002; Reynolds and 
Kamphaus 1992; Conduct 
Problems Prevention Research 
Group 1999 





Table 4 (continued) 



Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 




Experiences of Harassment at School (PA) 


EHS: .82, .85 and .87 for grades 3, 4 and 
5, respectively 


Flay, Acock, Vuchinich, and Beets 
2006** 


inclusion (e.g., 
individuals made to feel 
part of the school) 


School Climate and Interactions Survey 
(Cp) 


SCIS: .92 for staff and .93 for parents for 
Feelings of Belonging subscale, .87 for 
Staff Culture of Belonging subscale, .88 
for staff and .94 for parents for Parent 
and Staff Relations subscale 


Marshall and Caldwell 2007 




Survey of Perceptions of Oneself and 
School (E for C) 


SPOS: .75 post intervention 


Pinhas and Kim 2004 




Latent School Climate (ICPS) 


LSC: >.7 for items from Behavioral 
Assessment System for Children 
depending on wave and/or sample 
(current study VS norming sample); >.7 
for items from Parent Teacher 
Involvement Questionnaire 


Kumpfer, Alvarado, Tait and 
Turner 2002; Reynolds and 
Kamphaus 1992; Conduct 
Problems Prevention Research 
Group 1999 


Social systems (advisories, 
leadership teams, etc.) 


NR 






Democratic governance 
practices 


School Climate and Interactions Survey 
(Cp) 


.78 for Sense of Autonomy and Influence 
subscale 


Marshall and Caldwell 2007 


Positive leadership 


School Climate and Interactions Survey 
(Cp) 


SCIS: .88 for School Leadership subscale 


Marshall and Caldwell 2007 




Character Development Survey (CQ; C of 
C) 


CDS: .82 for students, .92 for parents and 
staff on School Expectations subscale 


Johns 1997 


Positive physical environment 


NR 






Positive academic environment 


School Climate and Interactions Survey 
(Cp) 


SCIS: .85 for Altruism subscale 


Marshall and Caldwell 2007 




Character Development Survey (CQ; C of 
C) 


CDS: .70-. 92 depending on reporter and 
subscale 


Johns 1997 
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Domain/Construct 


Scaled Measure/s with Highest Reported 
Reliability for the Construct (Acronym for 
Program Assessed with Instrument) 


Reported Reliability of the Scale/s or 
Subscale/s* 


Citations 


Parent-Community Level 
Outcomes 








Parenting skills 


Parenting Skills Scale (ICPS) 


PSS: >.70 for items from the Alabama 
Parenting Skills Questionnaire depending 
on sample and subscale 


Kumpfer, Alvarado, Tait and 
Turner 2002 




Family Relations Scale (ICPS) 


FRS: .87 for Beliefs about the Family 
Subscale 


Gorman-Smith, Tolan, Zelli, and 
Huesmann 1996 


Participation in school 


Enhanced Relationships Survey (CSC) 


ERS: .76 for Parent Involvement and 
Supportiveness subscale 


Developmental Studies Center n.d. 




Character Development Survey (CQ; C of 
C) 


CDS: .70 for parents and .82 for staff for 
Parent Involvement subscale 


Johns 1997 


Parent support of 
schooEprogram 


NR 






Community 
climate/ environment 


Survey of Teacher Attitudes (PA) 


STA: .89 for Perception of Neighborhood 
Facilities, .76 for Neighborhood Sense of 
Community, and .79 for Social 
Disorganization of Neighborhood 
subscales 


Flay, Acock, Vuchinich, and Beets 
2006** 



NR=Not Reported — the construct was measured with a scale in at least one study, but no information on reliability was reported. 

NM=Not Measured with a scale in the research reviewed. 

* All reliability statistics were reported as Cronbach’s alpha (internal consistency reliability), with the following exceptions: Social Skills Rating System, 
Assertion subscale (BEST, OC) — test-retest reliability; TerraNova Supera PCE) — Kuder-Richardson 20 coefficients; Commitment to School and Academic 
Performance Rating, parent measure (RHC) — correlation; and Coping Skills-Decision Making Questionnaire (SCPP-YA) — inter-rater reliability. 



**The scales used in evaluations of Positive Action have undergone additional development work. Information regarding those updated scales can be found in 
Flay and Dubois 2007. 





Validity. Validity indicates whether the construct one aims to measure is actually being 
measured. In contrast to their treatment of reliability, the studies reviewed here included little 
information bearing on the validity of measures: Among the 36 programs reviewed, studies of 5 
programs included information regarding the validity of their measures (the Heartwood Ethics 
Curriculum, LIFT, Raising Healthy Children, Open Circle, and the Topeka Character Education 
Initiative). In the studies of these programs, information on validity was limited to statements 
indicating that the authors had considered the validity of their measures, but without specific 
explanation of procedures to establish or verify validity. In addition to the studies of these five 
programs in which researchers directly discussed the validity of their measures, our review of 
additional sources (cited or suggested in studies of the 36 programs included in this analysis) 
showed that studies of 1 1 other programs had included outcomes measured with instruments for 
which validity had been established in prior studies. 

Single-Item Measures. In addition to the scaled measures presented in Table 3, our analysis 
of outcome measures also documented single-item measures that researchers reported using (see 
Appendix A). 

These single-item measures fall into three broad categories. First, disciplinary infractions, 
such as referrals to the principal, suspensions, and expulsions, are reported using either school or 
district administrative records or teacher or student reports. Second, teacher or administrative 
records are used to report student grades. Finally, multiple individual items are reported that may 
be related but are not reported as scales. In this situation, authors explicitly cite low reliability as 
a reason for reporting individual items, or they do not report whether the measures were intended 
for item-level analysis and reporting, or should be considered with other measures for analysis. 
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IV. SUMMARY AND CONSIDERATIONS FOR USING THIS REPORT 



The purposes of this study were (1) to document the constructs measured in studies of a 
delimited group of character education programs; (2) to develop a framework for systematically 
describing and assessing measures of character education outcomes; and (3) to provide a 
resource for evaluators to help identify and select measures of the outcomes of character 
education programs. The first section below summarizes the findings of our review of outcome 
measures in studies of 36 character education programs, which addresses the first two of these 
three objectives. The second section suggests considerations for future research on the outcomes 
of character education programs, as researchers identify and select measures of the outcomes of 
character education programs. While representative of measures used to evaluate character 
education outcomes in a delimited set of programs, the outcome measures reviewed are not 
necessarily the best ones for use in specific character education studies. 



A. SUMMARY OF REVIEW FINDINGS 

In documenting the constructs measured in studies of our sample of character education 
programs, we found that: 

• Researchers measured a wide variety of student-level outcomes in cognitive, 
affective, and behavioral domains. Among these, behavioral outcomes were assessed 
most often, in studies of 31 programs; affective outcomes were assessed for 28 
programs; and cognitive outcomes were assessed for 25. 

• Researchers assessed outcomes at other levels with less frequency than student-level 
outcomes. School-level outcomes were measured in studies of 16 programs; parent or 
community level outcomes were assessed for 14 programs; and teacher/administrator 
outcomes were assessed for 7 programs. 



Our framework for describing and assessing measures of character education outcomes calls 
attention to several points: 

• Researchers used direct and indirect assessments of student outcomes, and they 
solicited information about outcomes from students, teachers, administrators, and 
parents. 

• Measurement approaches to assess outcomes included scales and subscales, stand- 
alone items, and non-scaled measures. Studies of all but five programs employed 
scaled measures. 

• Among the 95 scaled measures employed across studies of 36 programs, 46 were 
newly developed for the study at hand; 32 had been developed prior to the study and 
were employed “off the shelf’ by the studies’ authors; an additional 17 scales were 
formed by adapting previously developed measures. 
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• Information on psychometric properties was available for 62 of 95 scales employed. 
Among these, 30 scales had demonstrated reliability over .70; 27 demonstrated mixed 
reliability; and 5 had reliability below .70. Information on validity was not available 
for individual measures, but studies of 16 programs either made general claims about 
their measures’ validity or used measures with previous research addressing validity. 



B. CONSIDERATIONS WHEN USING THIS REPORT 

This study does not represent a comprehensive examination of all outcomes measured in 
research on all character education programs. Still, it provides a framework for categorizing 
measures used in evaluations of character education programs and a resource for identifying 
potentially useful, existing measures as researchers plan future studies. It also highlights several 
caveats as researchers consider which outcomes to measure and how best to measure them. 



1. Considering Outcomes to Measure 

As our taxonomy of outcome measures indicates (Table 2), character education programs 
address a wide variety of outcomes in diverse domains (cognitive, affective, behavioral) and for 
different actors (student, teacher, school, parent, community). This classification system can 
serve as a point of departure for researchers considering which constructs, in which domains, and 
for which individuals or entities a particular character education program might be expected to 
affect. 

As in any research enterprise, studies of character education would benefit from articulation 
of a fonnal theory of change linking specific program components to the key outcome or 
outcomes that might result from them. Our review of research on this limited sample of character 
education programs leaves unclear whether, in their approach to measurement, researchers are 
beginning with a well-developed theory of change. This is evidenced, in part, by the relatively 
few programs that assessed knowledge of the concepts and content that the program sought to 
convey — the most proximal outcome and one which, logically, would precede changes in how 
students feel or behave. Studies of 5 of 36 programs assessed knowledge of program content (as 
detailed in the program-specific appendix tables). If the theory of change for a program is not 
well developed, our classification of outcomes may allow researchers to “work backward” from 
our categories to identify the outcomes they believe are most relevant for a particular program, 
and then to consider which program components might affect each of these outcomes. 



2. Considering How to Measure Outcomes 

In describing outcomes assessed and approaches to measurement, this review prompts 
researchers to consider several factors in selecting appropriate measures for assessing outcomes 
of character education programs. Depending on the outcome of interest, researchers should first 
detennine which type of assessment might be most appropriate — direct, indirect, or report and, in 
the latter case, who the most appropriate reporter might be. Next, a review of available measures 
for the outcomes of interest — whether they are scales, individual items, or non-scaled 
measures — can help researchers to avoid “reinventing the wheel.” At the same time, it can 
provide them with a sense of whether they are conceiving of the outcome in a way that is in line 
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with prior research. Such alignment is not necessarily a requirement, but consistency among 
researchers in the conceptualization and measurement of outcomes may support cross-study and 
cross-program comparisons and help to advance character education as a field. 

Future research on character education outcomes could also benefit from careful attention to 
alignment between the conceptualization and measurement of constructs. Reliability and validity 
demonstrate some level of this alignment. Our review, however, revealed two ways in which 
measurement methods may lack such alignment: (1) there may be misalignment between items 
in a particular scale (they do not “hang together”); and (2) there may be a mismatch between the 
domain or construct a measure actually captures and the domain or construct the researcher 
conceptualizes or, at least, discusses. The first problem is demonstrated in scales with low 
internal consistency reliability, and possibly among those scales for which reliability was not 
reported. Reference to measures with demonstrated reliability — such as those presented in Table 
4 — could help researchers to avoid the problem of misaligned scale items. The second problem is 
perhaps more difficult to document, as it arises primarily in the particular language that 
researchers use to discuss their findings. The outcome taxonomy and related measures presented 
here can serve as a resource for researchers seeking to articulate a clear theory of change and to 
properly align conceptualization and measurement of character education outcomes. 
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APPENDIX A 
PROGRAM TABLES 




TABLE A. la 



ACQUIRING ETHICAL GUIDELINES FOR INDIVIDUAL SELF GUIDANCE (AEGIS) 



Program name: 


Acquiring Ethical Guidelines for Individual Self Guidance (AEGIS) 


Program type: 


Modular 


Grade level: 


K-6 


Studies reviewed: 


Weed, S.E. “Weber School District Character Education Evaluation: Summary Report.” 
Salt Lake City, UT: The Institute for Research and Evaluation, 1995. 


Description: 


A broad-based character education program for grades K-6 designed to be used 
throughout the school year. The program employs a sequential five-step model of 
teaching and learning: stimulating interest, modeling the concept, integrating the concept, 
linking with parents, and applying in the real world. AEGIS focuses on character-related 
concepts, such as rights and responsibilities, care and consideration, fairness and justice, 
effort and excellence, personal integrity, and social responsibility. The 35-lesson 
curriculum is taught two to three days per week, with 20-30 minutes for each lesson. A 
separate teacher manual which includes reproducible student materials is designed for 
each grade level. Training is available. 
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TABLE A. lb 



PROGRAM NAME: ACQUIRING ETHICAL GUIDELINES FOR INDIVIDUAL SELF GUIDANCE (AEGIS) 



Student Outcomes 


Affective 


Behavioral Cognitive 


Constructs/ Components 


Measurement Information 




X 


Risk behaviors: substance use 


Student self reports on an unspecified instrument were used to assess alcohol, 
tobacco, and drug experimentation and use. 


X 


X 


Attitudes/motives: prosocial dispositions 

Attitudes/emotions: caring 

Knowledge: interpersonal knowledge, 
intrapersonal knowledge 

Reasoning: moral/ethical reasoning 


In this direct assessment of student learning of central character concepts, 
students were presented with a series of ethical dilemmas or scenarios 
(8 scenarios and 16 items for K-2; 20 scenarios and 40 items for grades 3-6) 
involving one or more of the concepts taught in the curriculum (care and 
consideration, social responsibility, ethical reasoning, personal responsibility, 
retaliation [avoidance], and respect for property). Each scenario was 
presented twice during the test, once with a behavioral response considered a 



negative indicator of character and once with a response considered a positive 
indicator of character. The children indicated their endorsement of the given 
response on a scale ranging from 1-5. Younger students responded by 
coloring segments of a thermometer with five divisions (corresponding to 1-5 
scale). Children were instructed that if they would do what the statement 
says, they should color the thermometer all the way to the top; if they would 
not do what the statement says, they should color only the lowest division of 
the thermometer; and if they were not sure, they should color somewhere in 
between. 
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TABLE A.lc 



PROGRAM NAME: ACQUIRING ETHICAL GUIDELINES FOR INDIVIDUAL SELF GUIDANCE (AEGIS) 

Other Outcomes 



Level 


Construct/ Components 


Measurement Information 


Teacher/Administration Level 




Staff morale 


Teacher self reports on an unspecified number of items related to teacher 
attitudes and behaviors related to their work. Item examples included the 
frequency with which the teacher had lost his/her temper with students 
(never, occasionally, or often); how he/she felt about being a teacher (very 
bad, okay, or very good); and the extent to which he/she had considered 
quitting (not at all, somewhat, or very seriously). 


Parent/Community Level 




Parent support of school/program 


Parent reports on an unspecified instrument were used to gauge support for 
the AEGIS program. In addition to a dichotomous yes/no item regarding 
support, parents were asked to rank the program in relation to other academic 
subjects (language arts, math, science, social studies, health, physical 
education, music, and art). 






TABLE A. 2a 



BUILDING DECISION SKILLS (BDS) 



Program name: 


Building Decision Skills (BDS) 


Program type: 


Modular 


Grade level: 


6-12 (12 studied) 


Studies reviewed: 


Leming, James S. “Integrating a Structured Ethical Reflection Curriculum into High 
School Community Service Experiences: Impact on Students’ Sociomoral Development.” 
Adolescence, vol. 36, no. 141, 2001, pp. 33-45. 


Description: 


Building Decision Skills aims to raise middle and high school students’ awareness of 
ethics, help them gain practical experience in developing core values, and give them 
practical strategies for dealing with ethical dilemmas. Building Decision Skills consists of 
10 lessons that can fill two consecutive weeks of daily lessons or be drawn out over a 
longer period. Using readings, handouts, and overheads, the teacher covers key concepts. 
Students are encouraged to think about the key concepts through small group activities, 
class discussions, and homework assignments. The program also includes school-wide 
components (such as group discussions, seminars, and assemblies). And it can be 
combined with service learning. 
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TABLE A. 2b 



PROGRAM NAME: BUILDING DECISION SKILLS (BDS) 
Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X 



X X 



X 



X 



Reasoning: moral/ethical reasoning 



Attitudes/motives: intrapersonal 
strengths 



Attitudes/motives: prosocial 
dispositions; civic dispositions 
Prosocial behaviors: positive 
participation 



Attitude/motives 

Attitudes/emotions 



To assess students’ sensitivity to the ethical issues, they were presented with three right - 
vs.-right conflicts (see a fellow student steal property from community service site; decide 
whether to continue to tutor elementary student or take after-school job; how to handle discovery 
that local homes sources of pollution) and asked to write responses. Responses were scored on 
the extent to which they demonstrate (1) ethical awareness (i.e., recognize an ethical issue 
exists), (2) ethical acceptance (i.e., who is responsible for the situation), and (3) an ethical 
perspective (i.e., use of language taught in BDS such as trust vs. loyalty or justice vs. mercy). In 
each of the three areas, students received a score of 3 (high), 2 (medium), or 1 (low). 

Students responded to the 20-item Janis-Ficld Feeling of Inadequacy Scale (Robinson and 
Shaver 1973) to capture their sense of confidence in social settings. Items addressed, for 
example, how often students worry about whether others like to be with them, feel inferior, or 
feel like they dislike themselves. Responses occurred on a 5-point Likert-type scale ranging 
from “very often” to “practically never.” 

Students responded to a Social Relatedness Questionnaire with items drawn from Newmann and 
Rutter’s (1983) Social Development Scale (SDS) . The seven-item Sense of Social Responsibility 
(General) subscale exactly matched items from the Non-School Responsibility subscale of the 
SDS and included items such as “I feel I should be doing something about problems in our 
community,” and “I’m concerned about how to make our community a better place for 
everyone.” The three-item Sense of Social Responsibility (School) subscale exactly matched the 
School Responsibility subscale from the SDS and included items such as “I try to get other 
students to support out school’s programs and activities, and “I try to imagine how we could 
improve the school for everyone.” Students responded to these items on a Likert-type scale 
ranging from “never true” (1) to “always true” (5). The final subscale addressing Future 
Community Participation was adapted from two subscales in Newmann and Rutter’s measure 
( Future Political Participation and Future Affiliation ), but specific items, the total number of 
items, and their response scale are not reported. 

Students rank-ordered 12 values including: compassion, responsibility, wealth, independence, 
honesty, recognition, respect, influence, pleasure, community, fairness, and intimacy. This task 
was modified from a task outlined in Rokeach 1973. 




TABLE A. 3a 



BUILDING ESTEEM IN STUDENTS TODAY (BEST) 



Program name: 


Building Esteem in Students Today (BEST) 


Program type: 


Comprehensive 


Grade level: 


K-8 (4-6) 


Studies reviewed: 


Gooding, T.F. “Character Education: Perceptions of Social Skills Acquisition in Two 
Elementary Schools.” UMI No. 3123558. Dissertation Abstracts International, vol. 65, 
no. 02, 2004. 


Description: 


According to Gooding (2004), Building Esteem in Students Today (BEST) emphasizes 
the development of regard for oneself, one’s school, and other students. The program, 
intended for children in kindergarten through eighth grade, focuses on nine themes: 
positive classroom, courtesy, responsibility, caring and sharing, goal setting, honesty, 
conflict and feelings, health and prevention, and esteem. For each theme, classrooms 
implement a 15- to 25-minute lesson emphasizing its importance. Additional instruction 
on the character trait can be included in the daily lesson plan. Monthly newsletters for 
staff and parents emphasize and reinforce the themes of the program throughout the year. 
The BEST program emphasizes an orderly environment and participation by 
administrators, counselors, parents, and others. The program was initially developed by 
the Institute for Human Resource Development (an Indiana nonprofit) under the auspices 
of the Indiana State Department of Education. 
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TABLE A. 3b 



PROGRAM NAME: BUILDING ESTEEM IN STUDENTS TODAY (BEST) 

Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X X 



X X 



Attitudes/emotions: school bonding 

Attitudes/motives: attitudes toward 
diversity 

Competencies/skills: responsibility, 
integrity, respect 

Attitudes/motives: caring 

Competencies/skills: intrapersonal 
competency, interpersonal competency 



Student self report using BEST Student Survey of School Connectedness 
(BEST 2000) with 13 items on school connections (e.g., have a role model), 
belonging, and learning positive behaviors in school (e.g., responsibility, 
patience); responses on 5-point scale of agreement (strongly disagree to 
strongly agree). 



Student self-report using the Social Skills Rating System (SSRS; Gresham 
and Elliott 1990), with differing items across a variety of subscales. Students 
responded to subscales for Cooperation (items on behaviors like 
organization, focus, attention), Assertion (items on behaviors like speaking 
up for self and proactively interacting with others in a group), Self-Control 
(items on behaviors like regulating behavior appropriately), Empathy 
(characteristics such as listening, demonstrating social overtures, 
understanding feelings). These subscales comprise the total Social Skills 
Scale (34 items; 10 Cooperation, 10 Assertion, 10 Self-Control, 10 Empathy; 
note that three items belong to more than one subscale). Students used a 3- 
point response scale of “never,” “sometimes,” or “very often.” 




TABLE A. 3c 



PROGRAM NAME: BUILDING ESTEEM IN STUDENTS TODAY (BEST) 

Other Outcomes 



Level 


Construct/ Components 


Measurement Information 


School/Class Level 




School climate 


Teacher report on the percentage of students perceived to demonstrate a certain 
frequency of competency in areas such as responsibility, decision making, 
conflict, and school enjoyment (10 items); responses were noted as the 
percentage demonstrating it on a 3-point scale of frequency (seldom, 
occasionally, or almost always). 




School climate 

Positive physical environment 


Principal report on 18 behavior problems at school (e.g., fighting, vandalism, 
truancy, disobey); responses on 3-point scale of frequency (low to high). 





TABLE A. 4a 



CARING SCHOOL COMMUNITY (CSC) 



Program name: 


Caring School Community (CSC) 


Program type: 


Comprehensive 


Grade level: 


K-6 (3-6 studied) 


Studies reviewed: 


Battistich, V. “Effects of a School-Based Program to Enhance Prosocial Development on 
Children’s Peer Relations and Social Adjustment.” Journal of Research in Character 
Education, vol. 1, no. 1, 2003, pp. 1-16. 




Battistich, V., E. Schaps, M. Watson, and D. Solomon. “Prevention Effects of the Child 
Development Project: Early Findings from an Ongoing Multi-Site Demonstration Trial.” 
Journal of Adolescent Research, vol. 11, no. 1, 1996, pp. 12-35. 




Roberson, I. “ Evaluation Report: Caring School Communities 2003-2004.” San 
Francisco, CA: San Francisco Unified School District, Program Evaluation & Research 
Department, 2006. 


Description: 


Caring School Community™ (CSC) is a modified version of a program formerly known 
as the Child Development Project. The CSC is a multiyear school improvement program 
that involves all students in grades K-6. 




The program aims to promote core values, prosocial behavior, and a schoolwide feeling 
of community. The program consists of four elements originally developed for the Child 
Development Project: class meeting lessons, cross-age “buddies” programs, “homeside” 
activities, and schoolwide community. Class lessons provide teachers and students with a 
forum to get to know one another, discuss issues, identify and solve problems 
collaboratively, and make a range of decisions that affect classroom life. Cross-age 
buddies activities pair whole classes of older and younger students for academic and 
recreational activities that build caring cross-age relationships and create a schoolwide 
climate of trust. Homeside activities, short conversational activities that are sent home 
with students for them to do with their parent or caregiver and then to discuss back in 
their classroom, incorporate the families’ perspectives, cultures, and traditions, thereby 
promoting interpersonal understanding. Schoolwide community-building activities bring 
students, parents, and school staff together to create new school traditions. 
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TABLE A. 4b 



PROGRAM NAME: CARING SCHOOL COMMUNITY (CSC) 
Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/Components 


Measurement Information 




X 




Competencies/skills: interpersonal competency 


Student report on peer acceptance using sociometric nominations. Students were presented with 
five situations and had to nominate three classmates they would want to be with. Situations were 
both academic (work on class project, help on school work) and non-academic (invite to birthday 
party, choose on sports team, get help from when sad). Responses were totaled for each student for 
total number nominations, number of reciprocated nominations, and opposite-sex nominations. 


X 


X 




Attitudes/motives: internalizing problems 
Attitudes/emotions: reflectivity 
Prosocial behaviors: justice/faimess, service 
Risk behaviors: discipline issues 


Student report on peer behavior using sociometric nominations. Students were presented with 12 
behaviors and asked to nominate classmates (no restriction on number) as fitting for those 
behaviors. Items included prosocial behaviors (helping, treating others fairly), negative behaviors 
(disobey school rules, act impulsively), competitiveness, assertiveness, and social isolation. 
Responses for each behavior were totaled and converted to a proportion of class. 


X 


X 




Attitudes/motives: intrapersonal strengths, 
internalizing problems 

Competencies/skills: interpersonal competency 


Student self renort using a survev of Intra/Internersonal Comnetencv. Items covered Social 
Dissatisfaction using the Child Loneliness Scale (Asher et al. 1984: Asher and Wheeler 1985). (16 
items; e.g.. “I have nobodv to talk to in class.”). Social Anxietv using Social Anxietv Scale for 
Children (LaGreca et al. 1988) (10 items; e.g., “I’m afraid that other kids will not like me.”). 
Perceived Popularity (8 items; e.g., “Other children like to play with me.”), Social Competence (17 
items; e.g., “I’m very good at working with other children.”), Self-Esteem (17 items; e.g., “I like 
myself just the way I am.”), and Liking of School (7 items; e.g., “I like my school.”). Response 
options are not reported. 




X 




Risk behaviors: substance use, violence, 
discipline issues, crime 


Student self report of risk behaviors, including use of cigarettes, alcohol, and marijuana on 5-point 
scale (l=never, 2=once or twice, 3=once in a while, 4=often, 5=used previously but not anymore) 
and frequency of delinquency (10 items; e.g., run away, steal, gang fight) on 5-point scale ranging 
from “never” to “10 or more times.” 


X 






Attitudes/emotions: school bonding 


Student self renort on Sense of Community Survev (Developmental Studies Center) with 45 close- 
ended questions falling in four areas including liking school, sense of classroom community, 
enjoyment of class, and trust in and respect for teacher. Response options are not reported. 






X 


Knowledge: academic content 


Student direct assessment on California Standards Test (CST; California Department of Education) 
measuring progress toward state academic content standards (percent of students achieving or 
exceeding proficiency in English Language Arts and Mathematics). 










Student direct assessment on the California Achievement Test. Sixth Edition (CAT/6)/TerraNova. 
The Second Edition (CTB McGraw-Hill), a norm-referenced test (comparing California students to 
those in other states) in Reading, Language, Spelling, and Mathematics. 




erv 



TABLE A. 4c 



PROGRAM NAME: CARING SCHOOL COMMUNITY (CSC) 
Other Outcomes 



Level 


Construct/Components 


Measurement Information 


Teacher/Administration Level 




Professional efficacy; staff morale 


Teachers completed an Enhanced Relationships Survev (Developmental Studies 
Center) with 51 close-ended questions. Subscales addressed Sense of Efficacy as a 
Teacher and Enjoyment of Teaching. Response options are not reported. 


School/Class Level 



School climate: interactions Teachers completed an Enhanced Relationships Survey (Developmental Studies 

Center) with 51 close-ended questions. Subscales addressed Trust in Students, 
Positive Relationships among Students, Positive Relationships Between Students 
and Teachers, Faculty Collegiality, and Principal Accessibility and Supportiveness. 
Response options are not reported. 



Parent/Community Level 




Participation in school 


Teachers completed an Enhanced Relationships Survev with 51 close-ended 
questions. Subscale addressed Parent Involvement and Supportiveness. Response 
options are not reported. 







TABLE A. 5a 



CHANGING LIVES (CL) 



Program name: 


Changing Lives (CL) 


Program type: 


Modular 


Grade level: 


K-12 (K-6 studied) 


Studies reviewed: 


Katsuyama, R.M., and C.E. Kimble. “Effects of the Changing Lives Character Education 
Program Upon Behaviors and Perceptions of Students, Teachers, and Parents: Evidence 
of Transformations in the School Climate.” Litchfield, MN: Mark I of North America, 
Inc, 2002. 


Description: 


Changing Lives is a K-12 character education program focused on instilling pillars of 
character such as respectfulness, honesty, and trust in participating students. Lessons and 
activities revolve around a “word of the week” that can be displayed on a poster in the 
classroom; there are 36 words of the week to address during the school year. A number of 
supporting materials are available. For the elementary grades. Changing Lives offers 
teacher guides, student workbooks and activity books, character journals and character 
posters. For middle and high school classes, Changing Lives offers journals, workbooks, 
and posters. Materials to use with athletic teams are also available. The program aims to 
increase student achievement, improve student behavior, improve student and staff 
attendance, improve staff morale, and increase parent and community engagement. 
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TABLE A. 5b 



PROGRAM NAME: CHANGING LIVES (CL) 
Student Outcomes 



Affective 


Behavioral Cognitive Constructs/Components 


Measurement Information 


X 


Competencies/skills 
Prosocial behaviors 
Risk behaviors 


Students responded to a Student Behavior Survev. Twentv-one items addressed 
behaviors such as honesty, completing work on time, and kindness to others. 
Twenty-six items addressed the frequency of positive behaviors directed toward 
others, the frequency of negative behaviors directed toward others, the frequency 
of positive behaviors received from others, and the frequency of negative 
behaviors received from others. Specific items are not provided, and the response 
options are unknown. 


X 


Attitudes/motives: attitudes toward 
school 


Students responded to five questions addressing the Perceived Value of Character 
Education (e.g., helped their motivation to leam, helped them become more 
cooperative). Students responded on a 5-point scale, though specific values are 
unknown. 




Risk behaviors 


Information was collected on the number of disciplinary incidents at school 
through monthly behavioral records. No additional information is provided. 



X 
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TABLE A. 5c 



PROGRAM NAME: CHANGING LIVES (CL) 
Other Outcomes 



Level Construct/Components Measurement Information 



Teacher/Administration Level 



Attendance Monthly information was collected concerning teacher attendance. No additional information is provided. 



School/Class Level 



Risk behaviors 



Monthly records on the number of school visitors were collected. No additional information is provided. 



School climate: 
collective 
norms/values; 
interactions 

Social systems 

School climate: 
collective 
values/nonns; 
interactions; inclusion 



In a questionnaire on Impressions of the School and Interactions Patterns , teachers responded to 50 items on the school, the staff, and 
communication and interaction patterns among them. An additional 13 items addressed the frequency of positive and negative behaviors 
and relations at school among students in the past year. There is no report of: specific items, the number of items addressing each 
construct, the scale for each item, or how items were summarized. 



In a Student School Climate Questionnaire , students responded to an unspecified number of items. Analyses identified six factors 
including: Classmate Citizenship (e.g., “Your classmates try to do their best in school”), Teacher Support (e.g., “Your teacher helps you 
when you need it”), Classroom Order/Control (e.g., “You/your classmates push or hit when angry”), Adult Expectations (e.g., “Your 
teacher expects you to learn a lot”), Prosocial Behaviors (e.g., “You/your classmates do helpful things for others in class”), and 
Inclusiveness (e.g., You/your classmates leave some students out when doing an activity). The response scale ranged from 1 to 5, though 
specific values are not provided. 



School climate: 
collective 
norms/values; 
interactions 



Teachers responded to an unspecified number of items in a Teacher School Climate Questionnaire . Analyses identified three 
factors: Student Recognition (e.g., “Are students regularly recognized for positive attitudes and behaviors?”), Professional Development 
(e.g., “Are you encouraged to develop your knowledge and teaching skills?”), and Student Academic Initiative (e.g., “Do your students 
view homework as an opportunity to show they have learned?”). Responses occurred along a 5-point scale, though exact values are not 
reported. 



School climate Students responded to an 18-item questionnaire on Behavioral Expectations at School : items addressed respect for others, honesty, 

completing work on time, kindness to others, etc.; sample items were not provided; responses occurred along a 5-point scale; response 
options are not indicated for the baseline, but for the follow-up survey, responses requested comparisons with the prior year, with values 
ranging from “much worse” to “much better.” 

At the follow-up survey, students also responded to questions indicating whether they lived up to school expectations for behavior. 



Uncertain 



In a telephone interview, parents responded to 26 items evaluating the child’s experience at school in the past year. Specific items, 
response options, and approach to summarizing items are not provided. 



Parent/Community Level 



Participation in school 



Monthly records were collected concerning parent attendance at parent-teacher conferences and other school events. No additional 
information is provided. 







TABLE A. 6a 



CHARACTER COUNTS! (CC!) 



Program name: 


Character Counts! (CC!) 


Program type: 


Comprehensive or modular 


Grade level: 


PreK-12 (1-12 studied) 


Studies reviewed: 


Walsh-Vettern, R., and B. Wright. “South Dakota Character Counts! Evaluation Results 
(First Year’s Data 1997-1998).” Brookings, SD: Cooperative Extension Service at South 
Dakota State University, 1999. 

Fruechte, K., and C.G. Mitchell. “South Dakota Character Counts! Evaluation Results 
(Year Five-2002).” Brookings, SD: Cooperative Extension Service at South Dakota State 
University, 2003. 

Harms, K., and S. Fritz. “Internalization of Character Traits by Those Who Teach 
Character Counts!” Journal of Extension, vol. 39, no. 6, 2001. 

Lennox School District. “Lennox Character Counts! Program. Final Performance Report 
Covering the Entire Performance Period 2002-03 Through 2005-06 School Years.” 
Lennox, CA: Lennox School District Department of Instructional Services, 2007. 


Description: 


Character Counts! is a framework based on a set of values referred to as the Six Pillars of 
Character: trustworthiness, respect, responsibility, fairness, caring, and citizenship. It is 
designed to be implemented as a comprehensive character education program, 
appropriate for all grades (pre-K-12), but can be implemented in a more modular way by, 
for example, integrating lessons or activities into one subject area. While most often 
implemented in schools, the program includes parent and community components. For 
example, parents and guardians are encouraged to attend and participate in CC! activities. 
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TABLE A. 6b 



PROGRAM NAME: CHARACTER COUNTS! (CC!) 
Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X X 



X 



X 



X X 

X 



X 



X 



Attitudes/emotions: school bonding 

C ompetencies/ skills 

Prosocial behaviors 

Knowledge: understanding 
values/norms 



Teachers responded to a series of open-ended questions (e.g., “Can you provide examples of where CC! 
made a difference in someone’s life?”); from those responses, researchers identified four categories of 
outcomes: children’s treatment of each other (e.g., “...students are complimenting each other more 
often...”), life skills competency (e.g., “...this is a way for [my students] to work on social skills and 
improve their ability to interact with others.”), a framework for behavioral corrections (shared values; e.g., 
“I think the students more clearly understand what is expected of them.”), and sense of 
community/belonging (e.g., “[My students] act much more as a community, which is great because we are 
so diverse.”). 



Attitudes/motives: prosocial 

dispositions 

Attitudes/emotions: justice, fairness 



In this measure of Attitudes Toward Ethical Issues , students in grades 7-12 responded to an eight-item 
questionnaire that addressed attitudes about pillars of character (4 items; e.g., “Sometimes it is okay to 
respond to an insult with force” and “It’s okay to lie or cheat if it’s to avoid unfair consequences”), about 
parents (1 item; e.g., “My parents set a high example of honesty and ethics”), and their World View (3 
items; e.g., “In today’s world, people often must lie or cheat to succeed” and “On a practical, real life basis 
honesty is the best policy”). Students responded on a Likert-type scale ranging from 1 to 5 that indicated the 
level of agreement. 



Competencies/skills: integrity 

Prosocial behaviors: trustworthiness; 
service; kindness 

Risk behaviors: substance use; 
discipline issues 



Knowledge: academic content 



Students in grades 7-12 responded to a survey regarding their Recent Behaviors in the past six months. 
Students in grades 6-8 responded to a survey regarding their behavior since being involved in CC! 
Individual items are not reported for the former survey, but authors report that the two surveys are similar. 
Question in the latter survey (grades 6-8) addressed Behavior Toward Rules (6 items; e.g., “Cheated on an 
exam or quiz.” “Missed class without a legitimate excuse.” “Failed to get my schoolwork done on time.”), 
Behavior Toward Others (7 items; e.g., “Used physical force against someone.” “Told a lie to a parent.” 
“Teased someone because of race or ethnicity.”), Illegal Behaviors (6 items; e.g., “Drank alcoholic 
beverages.” “Used an illegal drug.” “Defaced or vandalized property.” “Took something without 
paying.”), and Positive Behaviors (3 items; e.g., "Helped someone study for an exam.” “Volunteered to do 
something in the community.” “Told the truth though it might get me into trouble.”). In the survey for older 
students, they responded using five frequency categories (with 1 = no times; 2 = 1 time; 3 = 2-5 times; 4 = 
6-10 times; and 5=11+ times). In the survey for the younger group, they responded with the amount of 
behavior relative to prior to the CC! program; response options included “more often,” “the same,” “less,” 
and “never.” Items were summed by subcategory. 

Academic achievement of seventh grade students was measured with California Achievement Tests, 
Sixth Edition (CAT/6)/TerraNova, The Second Edition, and Stanford Achievement Test, Ninth Edition 
(SAT 9) scores. Both are nonn-referenced tests (comparing California students to those in other states). 
Additionally, the Academic Performance Index (API) measures the academic performance and growth of 
California schools on a variety of academic measures. 



Attitudes/motives 



Teachers graded students’ citizenship and work habits. No additional information is provided. 



Prosocial behaviors 

Prosocial behaviors' positive Student participation in extracurricular activities was reported, but no information on its source is provided. 

participation 




TABLE A. 6c 



PROGRAM NAME: CHARACTER COUNTS! (CC!) 
Other Outcomes 



Level 


Construct/Components 


Measurement Information 


Teacher/Administration Level 




Understanding values/norms 


In a survey of educators involved in CC!, respondents reported their level of sensitivity to ethical issues 
since teaching CC!. All items are phrased, “Since teaching Character Counts!, I am more sensitive to ethical 
dilemmas [in different levels of interaction].” Educators responded using a Likert-type scale ranging from 
“strongly disagree” (1) to strongly agree (5). 




Support 


In a survey of educators involved in CC!, respondents reported their behavior regarding advocacy for ethical 
decision making and taking a stand on ethical issues since teaching CC!. All items are phrased, “Since 

teaching Character Counts!, I am more likely to... Educators responded using a Likert-type scale ranging 
from “strongly disagree” (1) to strongly agree (5). 




Staff morale 


Students and teachers were asked whether school climate or staff/student morale had improved this year as a 
result of the Six Pillars of Character and to provide an example. 


School/Class Level 




School climate 


Students and teachers were asked whether school climate or staff/student morale had improved this year as a 
result of the Six Pillars of Character and to provide an example. 




School climate: collective 
norms/values; interactions; 
inclusion 


Teachers responded to 1 1 items regarding changes in students’ behavior after implementation of CC!. Items 
fell in three areas: behavior toward one another (fight less often; help each other more often; and call each 
other names less [grades 1-6] or treat each other better [grades 7-12]), behavior toward authority (students 
less disruptive in class; treat me with more respect; and are less destructive of property [grades 1-6] or have 
more respect for property [grades 7-12], and adherence to school rules (cheating, playing by the rules, lunch 
manners, recess behavior, and lunch behavior [grades 1-6]; or cheating, detention/suspension, skipping 
classes, being late for classes, and getting homework done [grades 7-12]). Teachers responded on a scale 
ranging from 1 to 5 indicating the level of agreement (with 3 = “unsure”). 


Parent/Community Level 




Parenting skills 


In a survev on Recent Behaviors, students in grades 6-8 responded to questions on behavior since being 
involved in CC! Three items addressed Parental Behaviors (“Was lied to by a parent.” “Discovered my 
parent doing something dishonest.” “Had a parent break a promise to me.”). Response options included 
“more often,” “the same,” “less,” and “never.” Items in the subcategory were summed. 




Participation in school 


Parent/guardian participation in school was measured by the number that had signed in at a variety of events. 
Head counts were taken when sign-in sheets were unavailable. 







TABLE A. 7a 



CHARACTER^ (CP) 



Program name: 


CHARACTERS (Cp) 


Program type: 


Comprehensive 


Grade level: 


K-12 (4, 8, and 11 studied) 


Studies reviewed: 


Marshall, J., and S. Caldwell. “Missouri Show Me CHARACTERplus Implementation 
Study.” St. Louis, MO: Cooperating School Districts, 2007. 


Description: 


CHARACTERPlus (Cp) is a grassroots initiative which began in St. Louis, Missouri, in 
1988. It is founded on the principle that successful initiatives are those that are holistic; 
school leadership teams are trained to facilitate parent, community, and school 
involvement activities that best fit their needs. Cp supports schools through the process of 
designing and implementing character education programs and recruiting and developing 
community support. The 10 “essentials” of the Cp process include: community 
participation, character education policy adopted by the school board, identified and 
defined character traits, integrating character education into the curriculum, experiential 
learning opportunities for students, evaluation, adult role models, staff development, 
student leadership, and sustaining the program. 
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TABLE A. 7b 



Program Name: C H A R A CT L R plus (Cp) 
Other Outcomes 



Level 


C onstruct/ Components 


Measurement Information 


School/Class Level 








School climate: collective norms/values; 
interactions; inclusion 

Democratic governance practices 

Positive leadership 

Positive academic environment 


Students, parents, and staff responded to the School Climate and Interactions Survey; the 
subscales contained in the survey differed slightly for each group. All three groups responded to 
the 12-item Feelings of Belonging subscale which included statements such as “Students are nice 
to each other” and the five-item School Expectations subscale which included statements such as 
“Students are expected to get along.” 

Students responded to three additional subscales on school climate. The five-item Sense of 
Autonomy and Influence subscale included statements such as “Students plan things together 
with their teachers.” The five-item Sense of Altruism subscale included statements such as 
“Students report having helped someone learn something.” The nine-item Feelings of 
Competence subscale included statements such as “Students like themselves the way they are.” 

Staff responded to a number of additional questions addressing staff-level school climate. Six 
items regarding Interactions with School Leadership captured aspects of teacher and 
administrator relationships (e.g., “Both the administrators and teachers take active roles in school 
activities” and “Things are well organized.”). Ten items addressed the Staff Culture of Belonging 
(e.g., “School staff members are supportive of one another” and “School staff share the same 
beliefs about the mission of the school.”). 

Parents and staff both responded to seven items on Parent and Staff Relations (e.g., “School staff 
members care about parents and their families” and “School staff members make parents feel 
welcome at school.”). Finally, parents responded to one item on School Quality, “Parents believe 
that their children get an excellent education at the school.” 

Response scales are not reported. 


Parent/Community Level 






Participation in school 


As Dart of the School Climate and Interactions Survey, students and parents responded to seven 
questions about Parent Involvement in school. Items included “Parents attend school functions” 
and “Parents talk with the student about what he/she is doing in school,” among other things. The 
response scale is not reported. 






TABLE A. 8a 



CHARACTER QUALITY (CQ) 



Program name: 


Character Quality (CQ) 


Program type: 


Comprehensive 


Grade level: 


Pre K-12 (K-5 studied) 


Studies reviewed: 


Chugach School District. “PCEP Grantee Report on Character Quality Program: 
Evaluation Report.” Anchorage, AK: 2004. 


Description: 


Character Quality (CQ) is a locally developed comprehensive program being 
implemented in four schools (PK-12) in one district in Alaska (in partnership with two 
others). CQ draws on and enhances the district’s success in systemic reform and attempts 
to link program goals to state educational standards. CSD focuses on implementation and 
integration of character qualities and education into district classrooms, home-school 
learning, and Anchorage House, a residential learning environment. The CQ project 
involves schools in remote rural communities located in south-central Alaska where 
poverty and unemployment are high and 21st century opportunities are low. 
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TABLE A. 8b 



PROGRAM NAME: CHARACTER QUALITY (CQ) 
Other Outcomes 



Level 


Construct/Components 


Measurement Information 


School/Class Level 




School climate: collective norms/values; 
interactions 

Positive leadership 


School climate was assessed with student (26 items), parent (37 items), and staff 
(46 items) responses to the Character Development Survev (Character Education 
Project). All three groups responded to 26 questions regarding the level of Kindness 
and Caring among students at the school (7 items; e.g., “The students at this school 




Positive academic environment 


are nice to each other” and “The students at this school try to include everyone”), the 
level of Respect and Responsibility (7 items; e.g., “The students at this school get 
along well together even if they are different” and “The students at this school insult 
or hit each other”), the level of Fairness and Honesty (7 items; e.g., “The students at 
this school tell the truth” and “The students at this school lie or cheat on their 
homework”), and School Expectations for Interpersonal Behavior (5 items; e.g., 
“Our school expects everyone to get along even if they are different” and “Our 
school expects everyone to be kind and caring”). 






Staff and parents responded to an additional seven items on Staff-Parent 
Relationships (e.g., “The school staff cares about the student’s families” and “The 
school staff treats parents with respect”) and one item on education quality 
(“Children get an excellent education at this school”). Note that the CDS includes an 
additional subscale on Staff Relationships that was not included in this study. 






Students responded on a 3 -point Likert-type scale (“almost always,” “sometimes,” 
and “hardly ever”), and staff and parents responded along a 5-point Likert-type 
scale. 


Parent/Community Level 




Participation in School 


Parents and staff responded to three items in the Character Development Survev 
(Character Education Project) regarding Parents’ Involvement in School (e.g., 
“Parents supervise their children’s homework” and “Parents help in the classroom”). 
Staff and parents responded along a 5-point Likert-type scale. 






TABLE A. 9a 



COMMUNITY OF CARING (C of C) 



Program name: 


Community of Caring (C of C) 


Program type: 


Comprehensive 


Grade level: 


K-12 (K-12 studied) 


Studies reviewed: 


Laing, S.O., K.D. Fink, and J.S. Johns. “Utah Community Partnership for Character 
Education Development Final Evaluation Report 1995-1999.” ERIC Document 
Reproduction Service No. ED463224. Salt Lake City, UT: Utah State Office of 
Education, 1999. 




Voelker, M.P. “An Option to ‘Just Say No’: Schools as Communities.” ERIC Document 
Reproduction Service No. ED9507505. Ann Arbor, MI: University of Michigan, 1994. 


Description: 


Founded in 1982 by Eunice Kennedy Shriver, Community of Caring was originally 
developed to prevent teen pregnancy by providing a comprehensive system of continuous 
care within a nurturing environment for pregnant adolescents. In its current form, the 
program aims to empower youth to be responsible and caring members of their 
community. 




The program is based on five core values: caring, respect, responsibility, trust, and 
family. A Community of Caring school is one with an active, participatory, caring 
program that includes all members of a school community. Program implementation 
involves staff development workshops on topics such as integrating values in and across 
the curriculum and expanding family and community involvement. Schools select a lead 
teacher or facilitator and develop a coordinating committee which can include students, 
staff, parents, and community members to oversee integration of Community of Caring 
values into the school and develop an action plan including goals, objectives, and 
activities for the program. Teachers integrate the five core values into the curriculum and 
daily activities. Schools also focus on developing student leadership, family and 
community involvement, and providing opportunities for service learning. 
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TABLE A. 9b 



PROGRAM NAME: COMMUNITY OF CARING (C of C) 
Student Outcomes 



Affective Behavioral Cognitive 


Constructs/ Components 


Measurement Information 


X 


Knowledge: understanding 
values/norms 


In this assessment of Student Knowledge of Character Education Concents, familiarity 
with values emphasized in character education was measured by asking students at each 
grade level to define and give an example of the school values. Teachers scored responses 
and reported the number of correct definitions and examples of behaviors representing 
desired values. 
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PROGRAM 



Level Constract/Components 

School/Class Level 

School climate: collective norms/values; 
interactions 

Positive leadership 

Positive academic environment 



School climate 



Parent/Community Level 



Participation in school 



TABLE A.9c 



: COMMUNITY OF CARING (C of C) 
Other Outcomes 



Measurement Information 



School climate was assessed with student (26 items), parent (37 items), and staff (46 items) responses to the 
Character Development Survey . All three groups responded to 26 questions regarding the level of Kindness and 
Caring among students at the school (7 items; e.g., “The students at this school are nice to each other” and “The 
students at this school try to include everyone”), the level of Respect and Responsibility (7 items; e.g., “The 
students at this school get along well together even if they are different” and “The students at this school insult 
or hit each other”), the level of Fairness and Honesty (7 items; e.g., “The students at this school tell the truth” 
and “The students at this school lie or cheat on their homework”), and School Expectations for Interpersonal 
Behavior (5 items; e.g., “Our school expects everyone to get along even if they are different” and “Our school 
expects everyone to be kind and caring”). 

Staff and parents responded to an additional seven items on Staff-Parent Relationships (e.g., “The school staff 
cares about the student’s families” and “The school staff treats parents with respect”) and one item on education 
quality (“Children get an excellent education at this school”). Staff responded to an additional nine items 
measuring Staff Relationships and the Degree of Endorsement of Various Values (e.g., “The school staff models 
the behaviors they expect of students” and “The faculty and administration work well together”). 

Students responded on a 3-point Likert-type scale (“almost always,” “sometimes,” and “hardly ever”), and staff 
and parents responded along a 5-point Likert-type scale. 

In the Community of Caring School Survey , students and staff responded to approximately 48 items (the number 
differed slightly for the two groups) addressing whether the school was upholding the values taught or expected 
in a Community of Caring school. The survey addressed leadership (e.g., “Administrators and teachers are 
committed to equity”), school climate (e.g., “There is an acceptance for the student who is different”), whether 
values were integrated into the curriculum (e.g., “There is a program that impacts the school environment by 
connecting values to the school organization”), and students’ opportunities to put learned values into practice 
(e.g., “Students feel comfortable participating in values discussions). Responses were given along a 5-point 
Likert-type scale ranging from “always happens” (1) to “never happens” (5). 



Parents and staff responded to three items in the Character Development Survey regarding Parents’ Involvement 
in School (e.g., “Parents supervise their children’s homework” and “Parents help in the classroom”). Staff and 
parents responded along a 5-point Likert-type scale. 






TABLE A. 10a 



CONNECT WITH KIDS (CWK) 



Program name: 


Connect with Kids (CWK) 


Program type: 


Modular 


Grade level: 


3-12 (3-12 studied) 


Studies reviewed: 


Page, B., and A. D’Agostino. “Connect with Kids: 2004-2005 Study Results for Kansas 
and Missouri.” Durham, NC: Compass Consulting Group, LLC, 2005. 


Description: 


Connect with Kids aims to promote prosocial attitudes and positive behavior of 
elementary (grades 3-5) and secondary (grades 6-12) school students by teaching core 
character values. Lesson plans include videos, story summaries, discussion questions, 
student games, and activities for both core and supplemental character traits. The 
classroom curriculum is reinforced by a website component and school-wide and 
community outreach components. The program can be incorporated into an existing 
curriculum or used as a standalone program. The school or teacher decides on the number 
of character traits covered in each session, so the program duration may vary from one 
semester to an entire academic year. 
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TABLE A. 10b 



PROGRAM NAME: CONNECT WITH KIDS (CWK) 
Student Outcomes 



Affective 


Behavioral 


Cognitive 


C onstructs/C omponents 


Measurement Information 




X 




Competencies/skills 
Prosocial behaviors 


In a questionnaire on Students’ Own Behavior, students responded to 12 items 
(for elementary school students) or 24 items (for middle/high school students) 
concerning their own behavior. Items included positive (“I am nice to students I do not 
know well”) as well as negative (“I make fun of other students”) statements. Responses 
occurred on a 5-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). 


X 


X 




Attitude s/ motives 
Attitudes/emotions 

Competencies/skills 

Prosocial behaviors 


Students were presented with eight scenarios in which a student or students enact a 
certain behavior. The first four scenarios included items such as, “A classmate can tell 
when she is getting angry and counts to ten before she says or does something.” 
Students indicated whether the person in the scenario was “very much,” “somewhat,” 
“just a little bit,” or “not at all” like them. 










The second four scenarios required students to select from four potential behavioral 
responses. For example, “If a classmate took something from me without asking, the 
first thing I would do is. . response options include (1) take it back, (2) go up to them 
right away and find out why they did it, (3) tell the teacher about the situation, and (4) 
think about why they may have taken it before doing anything. Potential responses 
were tailored to each scenario. The elementary, middle and high school versions of the 
questions are the same. 








Risk Behavior: discipline issues 


Teachers reported the number of behavior incidents by quarter that (1) were resolved in 
class, (2) the student was sent to the office or received detention, and (3) the student 
was suspended. No additional information is provided. 


X 








Teachers also evaluated each student’s behavior quarterly on a scale ranging from 1 to 
5 with higher numbers indicating greater severity (1 -‘never disruptive in class” to 
5=“almost always disruptive in class/disruptions were for serious and often violent 
behaviors like bullying, fighting, or destruction of property/student was often sent from 
class, sent to office, received detention, and/or was suspended or increasing levels of 
disciplinary action required.” 






X 


Academic content 


Teacher report of student quarterly grades using a table provided by the researchers. 
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TABLE A. 10c 



PROGRAM NAME: CONNECT WITH KIDS (CWK) 
Other Outcomes 



Level Construct/Components Measurement Information 



School/Class Level 




School Climate: collective values/norms; 
interactions; inclusion 


In a questionnaire on Perceptions of Students’ Behavior, students 
responded to 20 items (elementary school survey) or 32 items 
(middle/high school survey) and teachers responded to 37 items 
indicating the degree to which they agreed with statements about the 
behavior of students in the class. Items from all three questionnaires 
overlapped. Statements included both positive examples (“My 
classmates are nice to students they do not know well”) and negative 
(“My classmates make fun of other students”); responses on a 5-point 
scale ranging from 1 (strongly disagree) to 5 (strongly agree). 





TABLE A. 11a 



COOL KIDS (TRIBES AND KIDZLIT/K1DZMATH) 



Program name: 


COOL Kids (Tribes and KidzLit/KidzMath) 


Program type: 


Comprehensive 


Grade level: 


K-9 (K-6 studied) 


Studies reviewed: 


Bellevue Elementary School District. “PCEP Grantee Report on COOL Kids: Final 
Performance Report.” Santa Rosa, CA: Bellevue Elementary School District, 2007. 


Description: 


A comprehensive program, implemented in grades K-6 in three California counties (also 
K-9 in other counties), COOL Kids (Character education = Outstanding Outcomes in 
Learning) integrates a comprehensive reading- and math-focused character education 
program into an ongoing after-school program (COOL School). COOL Kids uses a 
combination of two branded character education programs, Tribes TLC and the 
KidzLit/KidzMath curricula. The program seeks to promote good character to build a 
sense of belonging and promote academic achievement. Teachers and staff are trained in 
the curricula to support positive modeling and content delivery. Service-learning clubs 
support positive character development. Character education curricula and concepts are 
also presented to parents enrolled in English as a Second Language classes. The program 
focuses on teaching tolerance, respect, responsibility, cooperation, honesty, and empathy 
through integrated programs, parent involvement, and teacher training. 




For Tribes, teachers organize their students into collaborative learning groups of three to 
six students, with each “tribe” working together to promote a spirit of cooperation and 
social acceptance. At the program’s core, students and teachers agree to honor four basic 
agreements while in the classroom: (1) they agree to listen attentively to one another, 
(2) they promise to show appreciation for one another and avoid “put downs,” (3) they 
promise to show mutual respect, and (4) they agree that all students have the "right to 
pass” on peer-led activities in which they would rather not participate. 




KidzLit/KidzMath curricula are designed soecificallv for use in out-of-school settings but 
can be implemented in the classroom. The program uses a five-part process to build and 
develop student’s appreciation for and knowledge of subject area content as well as “core 
values” including: helpfulness, fairness, personal responsibility, perseverance, self- 

respect, and respect for others. 
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TABLE A. lib 



PROGRAM NAME: COOL KIDS (TRIBES AND K1DZLIT/KIDZMATH) 

Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/ Components 


Measurement Information 






X 


Knowledge: academic content 


Student direct assessment on California Standards Test (CST; California 
Department of Education) measuring progress toward state academic 
content standards. 










Teacher and parent report; three items on achievement (reading, math, and 
homework); responses noted as assessing student as “improving or having 
reached grade level.'’ 








Prosocial behaviors: positive 
participation 


Administrative data on school attendance. 




X 




Risk behaviors: discipline issues 


Administrative data on suspensions. 


X 


X 




Competencies/skills: respect; 
intrapersonal competency; 
interpersonal competency 

Prosocial behaviors: positive 
participation 


Teacher and parent reports on student behavior. Teachers responded to four 
items addressing classroom behavior, willingness to participate in class, 
attention in class, and respect for others. Teachers were asked to assess 
students as “improving or having reached grade level after participating in 
COOL Kids.” Parents responded to two items on behavior and social skills. 
They were asked to assess students as “improving after participating in 
COOL Kids.” 


X 


X 




Attitudes/motives: intrapersonal 
strengths 

Competencies/skills: intrapersonal 
competency 


Teacher report on attention in class and self-confidence. Teachers were 
asked to assess students as “improving or having reached grade level after 
participating in COOL Kids.” 




TABLE A. 12a 



EDUCATING FOR CHARACTER (E FOR C) 



Program name: 


Educating for Character (E for C) 


Program type: 


Comprehensive 


Grade level: 


K-12 (3-5 and 7-8 studied) 


Studies reviewed: 


Pinhas, S., and W. Kim. “Evaluation Report: Denver Public Schools Education for 
Character; Non-Case Study.” Denver, CO: RMC Research Corporation, 2004. 


Description: 


The Denver Public Schools’ Educating for Character program aimed to create schools 
that would support children’s development as ethical, caring, and responsible young 
people. DPS used Educating for Character to build an infrastructure to support schools in 
planning, implementing, and strengthening their own character education programs. 
Educating for Character staff worked with schools to implement a consensus building 
process in which schools identified core values and created a shared vision around those 
values (i.e., a touchstone) and, from that, an implementation plan. Teachers and staff 
received ongoing coaching from Educating for Character stuff to support program 
implementation. 




Activities included teacher inservice on: caring, listening, knowing students, fairness and 
respect, promotion of enthusiasm and motivation for learning, and the importance of 
teachers exhibiting touchstone qualities; workshops on best practices in service-learning, 
leadership and parent education; and advice in program selection. 
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TABLE A. 12b 



PROGRAM NAME: EDUCATING FOR CHARACTER (E for C) 
Student Outcomes 



Affective 


Behavioral 


Cognitive Constructs/Components 


Measurement Information 


X 


X 


Attitudes/motives: prosocial 
dispositions; attitudes toward school 

Attitudes/emotions: school bonding 
Prosocial behavior 


For elementary school students, self report using Survey of Perceptions of 
Oneself in School and School Climate; 28 items on a variety of topics. All but 
three items addressed student-level outcomes/perceptions on topics including: 
school engagement (2 items), responsibility for learning (3 items), respect 
(3 items), caring (3 items), being heard (2 items), social responsibility (3 items), 
resilience (3 items), prosocial behavior (3 items), school safety (3 items); 
responses occurred on 4-point scale ranging from “never” to “always”. Items 
were analyzed individually. 


X 


X 


Attitudes/motives 
Attitudes/emotions 
Prosocial behaviors 
Competencies/skills 


For middle school students. 57 items in the Survev of Perceptions of Oneself 
and School addressing feelings/nerceptions of themselves, other students, their 
teachers, and their schools. Analyses identified the following subscales: School 
Engagement, Responsibility for Learning, Attitudes toward Learning, Caring, 
Respect, Expressing Self, Being Heard, Resilience, Altruism, and Safety. The 
number of items in each subscale is unclear. Due to low internal consistencies, 
items included in the Caring and Safety subscales were analyzed individually. 
Responses occurred on a 4-point scale ranging from “strongly disagree” to 
“strongly agree.” 
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TABLE A. 12c 



PROGRAM NAME: EDUCATING FOR CHARACTER (E for C) 
Other Outcomes 



Level 


Construct/Components 


Measurement Information 


School/Class Level 




Democratic governance practices 
School climate: inclusion 


Principals reported on Student Involvement in Service Learning using four items in 
a self administered email survey; topics ranged from students’ enthusiasm for 
service learning to their involvement in projects; responses ranged from “strongly 
agree” to “strongly disagree.” 




School climate: collective norms/values 


For elementary school students, self report using Survey of Perceptions of Oneself 
in School and School Climate; 28 items on a variety of topics with one subscale 
addressing school climate (3 items); responses on 4 point scale ranging from “never” 
to “always.” Items were analyzed individually. 




School climate: collective norms/values; 
interactions; inclusion 


For middle school students, 57 items in the Survev of Perceptions of Oneself and 
School addressing feelings/oerceptions of themselves, other students, their teachers, 
and their schools. See “Student Outcomes” table above for description. 


Parent/Community Level 




Participation in school 


Principals reported on Parent Involvement; there were an unspecified number of 
items in the self administered email survey; topics ranged from percentage of 
parents participating in school events this year to the willingness of parents to 
volunteer for school activities. Responses were open ended. 






TABLE A. 13a 



GIRAFFE HEROES 



Program name: 


Giraffe Heroes 


Program type: 


Comprehensive or modular 


Grade level: 


K-12 


Studies reviewed: 


Center for Health Education and Research. “Giraffe Project/Standing Tall Evaluation: 
Final Report 1995-1997.” Battle Creek, MI: W. K. Kellogg Foundation, 1991. 


Description: 


The Giraffe Heroes program is a modular, story-based curriculum focused on fostering 
caring, courage, responsibility, and civic engagement in students, grades K-12, and to 
help children identify real heroes (rather than celebrities). As a central component of the 
program, children learn about Giraffe Heroes, people identified as heroes for having 
taken on tough problems. The program offers age-graded curricula, as well as curricula 
for gifted and talented classrooms, and for service learning projects. The curricula for 
grades K-2, 3-5, and 6-9 provide lesson plans, stories of Giraffe Heroes, handouts, audio- 
visual materials, and materials that can be taken home to their families. For children in 
grades 10-12, the program offers a book with stories of Giraffe Heroes, concepts for 
becoming an active citizen, action planning tools, reflection questions, and inspiring 
quotations, along with a resource guide for implementing the program. Teachers can 
combine character education with service learning by giving students the opportunity to 
design and carry out a service project on an issue of their choice. 
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TABLE A. 13b 



PROGRAM NAME: GIRAFFE HEROES 
Student Outcomes 



Affective 


Behavioral Cognitive 


Constructs/Components 


Measurement Information 


X 


X 


Affect (general) 
Behavior (general) 


Student self report on single item on the degree to which they observed positive attitude or 
behavior changes in themselves as a result of the program. Response options on a Likert-type 
scale, ranging from “no changes” (1) to “many changes” (4). Respondents asked to give 
examples of the changes. 




X 


Knowledge: understanding 
values/norms; interpersonal 
knowledge 


Teacher report on five items regarding student competencies with respect to issues addressed 
in the curriculum (e.g., ability to identify and understand four key qualities of a Giraffe Hero 
[i.e., courage, caring, taking helpful action, persistence]; ability to distinguish a Giraffe Hero 
from a celebrity; ability to distinguish between a Giraffe Hero and a nice person; ability to 
identify Giraffe Heroes in their neighborhood, school, or community). Response options on a 
scale ranging from “low” (1) to “High” (5). 




X 


Reasoning: moral/ethical 
reasoning 


Students named one of their heroes and selected a reason for choosing that person from 14 
options, which researchers characterized as “Giraffe Characteristics” (e.g., courage, cares for 
others, takes responsibility, trustworthy) or “Other Characteristics” (e.g., talented, famous, 
popular, rich). 


X 


X 


Attitudes/motives: civic 
dispositions 

Reasoning: moral/ethical 
reasoning 


Students responded to two statements (“Violence is a big problem in many American 
communities” and “Air and water pollution in the United States continues to be a big 
problem”). Students reported level of agreement with four possible responses to these 
problems (i.e., that it “will always be a problem and there is nothing I can do about it;” “I 
feel it is my personal responsibility to help solve” the problem; it “should be left up to 
experts;” and “I feel my friends and I can make a difference”). Response options ranged 
from “strongly disagree” to “strongly agree.” 


X 


X 


Attitudes/motives: prosocial 
dispositions; attitudes toward 
diversity 

Reasoning: moral/ethical 
reasoning 


Students directly assessed through responses to three ethical dilemmas. Students were 
presented with the dilemma then asked to select their “most likely” response. Response 
options were meant to reflect different degrees of alignment with GH principles, specifically: 
build a cooperative solution; build a personal solution; avoid the problem; join others to 
worsen the problem; or “other.” Dilemmas included vignettes in which: (1) “The coach 
announces only students with passing grades can play, but you are failing math.” (2) “A 
family from another country moves into the neighborhood and opens a clean, friendly, well- 
stocked grocery store. Many kids in the neighborhood have been rude to the family and 
many adults won’t shop at the store.” (3) “A new kid in class looks and acts different 
(clothes, haircut, quiet, reads science-fiction). Many kids make fun of him but the teacher 
likes him because he is smart.” Response options were tailored to each scenario, e.g., in the 
first, options included “study to catch up,” “quit the team,” “trash the coach,” or “start a 
study group.” 
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TABLE A. 13c 



PROGRAM NAME: GIRAFFE HEROES 
Other Outcomes 



Level 


Construct/ Components 


Measurement Information 


Teacher/Administration Level 




Staff morale 


Teacher self report on single item on the degree to which they observed positive 
attitude or behavior changes in themselves as a result of the program. Response 
options on a Likert-type scale, ranging from “no changes” (1) to “many changes” 
(4). Respondents asked to give examples of the changes. 


School/Class Level 




School climate 

Democratic governance practices 


Teacher reports, responding to questions regarding the supportiveness/ flexibility of 
the school environment for GH implementation (e.g., whether required to obtain 
administrative approval to implement the curriculum) and teacher rating of the 
school environment on a continuum of curricular flexibility (ranging from 
“established curriculum that doesn’t change much” [1] to “progressive 
curriculum/always trying new things” [5]). 


Parent/Community Level 




Participation in school 


Teachers reported responses to questions or rated statements about community 
involvement in the school (e.g., whether utilized classroom volunteers and degree of 
impact of volunteers on students; whether the classroom formed a partnership with a 
local business and the quality of that partnership experience; whether the GH 
program resulted in more interactions between the community and the 
classroom/school than would have occurred without the program). Response options 
varied, depending on the question (e.g., yes/no; Likert-type scale ranging from 
“strongly disagree” [1] to “strongly agree” [5]). 







TABLE A. 14a 



HEARTWOOD ETHICS CURRICULUM FOR CHILDREN (HECC) 



Program name: 


Heartwood Ethics Curriculum for Children (HECC) 


Program type: 


Modular 


Grade level: 


PreK-6 


Studies reviewed: 


Leming, James S., Astrid Henricks-Smith, and James Antis. “An Evaluation of the 
Heartwood Institute’s An Ethics Curriculum for Children'. Final Report.” Paper presented 
at the annual meeting of the American Educational Research Association, Chicago, 1L, 
1997 (revised April 2000). 




Leming, James S. “Tell Me a Story: An Evaluation of a Literature-Based Character 
Education Programme.” Journal of Moral Education, vol. 29, no. 4, 2000, pp. 413-427. 


Description: 


The Heartwood Institute’s Ethics Curriculum for Children (HECC) is an interactive, 
literature -based curriculum for elementary school students (middle school curriculum is 
also available). The program focuses on seven attributes of good character, which 
developers have identified as universally salient: honesty, love, loyalty, courage, respect, 
justice, and hope. Lessons and home assignments are organized around multicultural 
stories. Program activities are designed to connect the experiences of characters in the 
stories to students’ own lives. The curriculum also includes optional components that 
allow for integration of character education themes across academic subject areas, as well 
as parental involvement. The curriculum includes three kits, each with 14 books; ideally 
there are 14 lessons taught over the course of a year (two lessons for each of the seven 
attributes), each lasting 30 to 45 minutes. The curriculum also includes a wall map to 
mark the location of each story, an effort to support multiculturalism. 
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PROGRAM NAME: HEARTWOOD 



Affective Behavioral Cognitive Constructs/Components 

X Knowledge: understanding 

values/norms 



X Knowledge: academic content 



X 



Attitudes/motives: prosocial 
dispositions 



A. 14b 



rHICS CURRICULUM FOR CHILDREN 
Outcomes 

Measurement Information 

Ethical Understanding Scale directly assessed students on understanding of the 
curriculum’s seven core attributes by (a) choosing the term that matches the example 
definition (grades 1-3) or (b) providing a written definition to complete a sentence 
(grades 4-6). For younger children, the measure included 14 items (two for each 
attribute; e.g., “When I share my favorite things with others, that is: hope or love”; 
“When I am nice to children who look different from me, that is: respect or hope”). 
Items were scored as correct (1) or incorrect (2). For older children, the measure 
included seven items (one for each attribute; e.g., “A person is loyal when...”; “A 
person shows courage when...”). Items were scored as showing no understanding (1), 
partial understanding (2), or full understanding (3). Scoring rubrics contained example 
responses for each attribute. 

Students’ geographic knowledge directly assessed. They were shown a black-line 
world map and asked to identify locations from the stories. For younger students 
(grades 1-3), six locations were read aloud and students placed a sticker on the map; 
older students (grades 4-6) were given a map and asked to number the 14 different 
story locations. 

Ethical Sensibility Scale , for which students reported responses to 20 “I” statements on 
whether they would act based on the seven core attributes (e.g., “If I see someone 
being mean 1 will tell them to stop”). Students in grades 1-3 were read the statement 
and then placed a colored sticker on a line representing a stop light — red for no, yellow 
for not sure, and green for yes (scored 1, 2, 3 respectively). Students in grades 4-6 
were given a questionnaire and instructed to circle the level of agreement on a 5-point 
scale (l=disagreement). Scores were summed across all items, ranging from 20 to 60 
for younger children and from 20 to 100 for older children, with higher scores 
indicating higher ethical sensibility or valuing of the ethical attributes (e.g., courage, 
respect). 
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TABLE A. 14b ( continued ) 



Affective Behavioral Cognitive Constructs/Components 



X 



Attitudes/motives: attitudes toward 
diversity 



X Prosocial behaviors 



X Risk behaviors: discipline issues 



X 



Competencies/skills: integrity 



Measurement Information 



A variation of the Social Distance Task (Koslin et al. 1969) was used to measure 
ethnocentrism. Children were shown 10 head-and-shoulder pictures of same-age 
children (general, not known peers), each picture on a separate page. The picture 
represented a girl or boy from one of five cultural groups (White, African-American, 
Hispanic, Asian, and American Indian). The child was asked to show their first 
impression of whether he/she would like the child in the picture as a friend. Children 
then had to place a sticker (grades 1-3) or an “X” (grades 4-6) on the paper, with 
proximity indicating the extent to which they would like to be friends with the children 
pictured. The distance from the mark to the picture was measured in centimeters with a 
maximum potential of 14.4. The distance across the 10 pictures were summed for a 
score ranging 0-144, with higher scores indicating a higher degree of ethnocentrism. 

Ethical Behavior Rating , for which teachers rated students’ “character-related” 
behavior on a 5-point scale; items were summed for a potential score of 16-80, with 
higher scores indicating better behavior. A variation of the student-level instrument is 
available from the developer website, also for use by teachers in the classroom. This 
instrument includes the same number of items but the response scale is based on 
number of children displaying the behavior (from 1 = “almost no one,” to 5 = “almost 
everyone”). The 16 items include both positive (e.g., “truthful,” “helpful toward 
others”) and problem behaviors (e.g., “puts down other children,” “initiates physical 
aggression”). 

Comparison of school administrative records of principal referrals for two consecutive 
years. 

Children were assessed for cheating behavior using an adaptation of the Hartshorne 
and May (1928-1930) improbable achievement test (Squares Puzzle). Children were 
instructed to trace a line between each of five sets of concentric squares with their eyes 
closed. Possible scores ranged from 1-5; a score higher than 1 correct line was 
classified as cheating. 




TABLE A. 15a 



I CAN PROBLEM SOLVE (ICPS) 



Program name: 


I Can Problem Solve (ICPS) 


Program type: 


Modular 


Grade level: 


PreK-6 


Studies reviewed: 


Kumpfer, K.L., R. Alvarado, C. Tait, and C. Turner. “Effectiveness of School-Based 
Family and Children’s Skills Training for Substance Abuse Prevention Among 6- to 8- 
Year-Old Rural Children.” Psychology of Addictive Behaviors, vol. 16, no. 48, 2002, 
pp. S65-S71. 


Description: 


I Can Problem Solve (ICPS) is a modular, school-based intervention that focuses on 
enhancing problem-solving and critical thinking skills in preschool and elementary 
school-age children. Children are trained to generate alternative solutions to interpersonal 
problems, consider the consequences of those solutions, and recognize thoughts, feelings, 
and motives that generate problem situations. According to the program developer, the 
intervention lasts approximately three months with 40 to 50 lessons and works with small 
groups (6-10) of children. (In the Kumpfer study, the program was implemented in 83 20- 
minute lessons.) The first 10-12 lessons focus on basic skills and problem solving 
language (e.g., “if... then” statements to emphasize that actions have consequences; 
“same/different” to emphasize multiple solutions). In the next 20 lessons, students learn 
to identify their own and others’ feelings in problem situations and to recognize that they 
can influence others’ responses. In the last 15 lessons, students use role-playing to 
practice their problem solving skills. Lessons can be integrated into all aspects of the 
classroom curriculum. The program was originally developed for four- and five-year-old 
children but has since been adapted for all elementary grades. Note that in the study 
reviewed here, ICPS was compared to family skills training program. 



A. 41 




A.42 



TABLE A. 15b 



PROGRAM NAME: I CAN PROBLEM SOLVE (ICPS) 
Student Outcomes 



Affective Behavioral Cognitive Constructs/Components Measurement Information 





X 


Competencies/skills: interpersonal 
competency, leadership 


Student, parent, and teacher reports on three subscales (Adaptability, 
Leadership, and Social Skills) of the Behavior Assessment for System for 
Children (BASC; Reynolds and Kamphaus 1992), were used to measure social 
competence. The full BASC instruments for parents and teachers provide 
comprehensive measures of adaptive and problem behaviors. The child version 
is appropriate for children aged 6 to 11 (preschool/ages 2 14 to 5 and 
adolescent/ages 12 to 18 version are also available). Parent and teacher 
respondents report how often certain behaviors occur on a 4-point Likert-type 
scale of ranging from “never” to “almost always.” 


X 




Competencies/skills: intrapersonal 
competency 


A Self Regulation Scale was derived using parent and teacher reports of child 
behaviors on the Impulsivity, Hyperactivity, and Aggressive/Disruptive 
Behavior subscales of the Parent Observation of Classroom Adaptation- 
Revised (POCA-R) and the Teacher Observation of Classroom Adaptation- 
Revised (TOCA-R; Werthamer-Larsson et al. 1990; Werthamer-Larsson et al. 
1991). The POCA-R and TOCA-R are structured interviews, in which parents 
and teachers report on the child’s behaviors that may affect adaptation to 
school. Responses are on a 6-point scale (“almost never” to “almost always”), 
with higher scores reflecting higher levels of aggression, hyperactivity, and 
problems with concentration. 
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TABLE A. 15c 



PROGRAM NAME: I CAN PROBLEM SOLVE 
Other Outcomes 



Level 


Construct/ Components 


Measurement Information 


School/Class Level 




School climate: interactions, inclusion 


Researchers derived a Latent School Climate Scale through a combination of 
measures from three other instruments: (1) student reports on the Attitude Toward 
School (9 Items) and Attitude Toward Teacher (10 items) subscales of the Behaviors 
Assessment for Children (child-report version; Reynolds and Kamphaus 1992); 

(2) an unreported number of items from the Parent Report on School Climate; and 

(3) parent (and/or teacher) reports of interactions with the school on the 26-item 
Parent and Teacher Involvement Questionnaire (Conduct Problems Prevention 
Research Group 1991). The study provides no additional information on how 
researchers combined these measures (i.e., which items were included or what 
analytic technique was used to develop the scale). 


Parent/Community Level 




Parenting skills 


Researchers derived a Parenting Skills Scale using parent self reports on the 
Parenting Practices Scale (PPS; Loeber et al. 1998) and the Alabama Parenting 
Questionnaire (APQ; Shelton et al. 1996). Items from all five subscales of the PPS 
were used (Extent of Involvement, Supervision and Rule, Positive Parenting, Effect 
of Discipline and Discipline Avoidance). The APQ is a 35-item parent survey 
involvement with the child (10 items), use of positive reinforcement (6 items), 
monitoring and supervision of the child (10 items), consistency in applying 
discipline (6 items), and the use of corporal punishment (3 items). Items are rated on 
a 5-point frequency scale ranging from 1 (never) to 5 (always). The study provides 
no additional information on how researchers combined these measures (i.e., which 
items were included or what analytic technique was used to develop the scale). 




Parenting skills 


Parent self report on an adapted version of the Familv Relations Scale (including 35 
items on Family Beliefs, Family Cohesion, and Family Structure; Gorman-Smith et 
al. 1996) used to gauge participants’ family relations. Responses were given on a 5- 
point Likert-type scale measuring the respondent’s belief in how true the item was 
for his or her family. 






TABLE A. 16a 





INSTITUTE FOR CHARACTER EDUCATION (ICE) 


Program name: 


Institute for Character Education (ICE) 


Program type: 


Comprehensive 


Grade level: 


K-12 (K-12 studied) 


Studies reviewed: 


Cl Associates. “Institute for Character Education: 2005-2006 Annual Report.” Costa 
Mesa, CA: Institute for Character Education, 2006. 

Schneider, Stephanie H., and Doug Grove. “Orange County Department of Education’s 
Institute for Character Education: Final Evaluation Report: 2003-2004, 2004-2005, 
2005-2006.” Costa Mesa, CA: Institute for Character Education, 2007. 


Description: 


The Orange County (Los Angeles) Department of Education’s Institute for Character 
Education (ICE) represents a formalized system of professional development of teachers 
and administrators, training them as “Character Education Fellows” to integrate character 
education into the school’s curriculum. Three aspects of character are at the program’s 
core: integrity (academic honesty, honesty, fairness, trustworthiness), respect (for others, 
self, and showing compassion), and responsibility (dependability, perseverance, and civic 
mindedness). From 2002 through 2006, the ICE included staff from public and private 
schools, YMCA after-school programs, and alternative education sites. The training 
featured a 5-day institute during the summer. Monthly trainings occurred then throughout 
the school year (2003-04, 2004-05). Activities at the summer institutes included: 
developing character education knowledge among staff, specifying character definitions 
(i.e., integrity, respect, and responsibility), specifying the ICE framework concerning 
cognitive, emotional, and behavioral aspects of character knowledge, developing the 
lesson plan format and classroom practices, and addressing family involvement. Monthly 
meetings included review lessons, moral dilemma discussions, and collaborating on ideas 
and resources. Fellows also worked with each other within the school to develop ICE 
action plans (e.g., family nights, establishing a character corner), and in the third year of 
the program, held their own monthly meetings. Parent information sessions were also 
held. Lessons are available on the ICE website. 
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TABLE A. 16b 



PROGRAM NAME: INSTITUTE FOR CHARACTER EDUCATION (ICE) 

Student Outcomes 



Affective 


Behavioral 


Cognitive Constructs/Components 


Measurement Infonnation 


X 


X 


Attitudes/motives: 
intrapersonal strengths 

Competencies/skills: 

respect 


Teachers responded to three items rating students’ Respect for Self and Others (respects others, even if 
disagrees; exhibits strong respect for self; has talked back to teachers or school staff) on a 5-point Likert- 
type scale ranging from strongly agree to strongly disagree. 


X 


X 


Attitudes/motives: prosocial 
dispositions 

Competencies/skills: 
respect; integrity 


Students responded to a survev of Integrity and Respect (Think. Do and How Manv Times! that 
addressed both attitudes and behaviors. Students in fourth grade received questions developed for the 
studv. Students in middle and high school responded to auestions drawn from the Character Education 
Survev (ETR Associates). In both instruments, subscales are referred to as “Think.” “Do.” and "How 
Many Times.” 






Prosocial behaviors: 
trustworthiness 


For the Integrity subscale, fourth graders received seven Think items (e.g., To do well in school, 
you sometimes have to cheat) with response options of agree, don’t know, disagree; two Do items (e.g., 
If make a mistake, I admit it) answered as agree, don’t know, or disagree; and five How Many items 
(e.g., taken money or something that did not belong to you) rated for their occurrence in past year as 
never, 1-2 times, or more than 3 times. Middle and high school students received seven Think items 
(e.g.. If I accidentally broke a window, I would tell, even if no one saw me) rated as strongly agree, 
agree, undecided, disagree, strongly disagree; three Do items (e.g.. If make a mistake, I admit it) rated as 
strongly agree, agree, undecided, disagree, strongly disagree; and four How Many items (e.g., taken 
money or something that did not belong to you) rated for past year with 0 times, 1-2 times, 3-4 times, or 
5 times or more. 








For the Respect subscale, fourth graders received two Think items (e.g., I respect people, even if 
they don’t agree with me) with response options of agree, don’t know, disagree; eight Do items (e.g., If 
someone hurts my feelings, I try to hurt him or her back) answered as agree, don’t know, or disagree; 
and 10 How Many items in which they rated the frequency of occurrence in the past year as never, 1-2 
items, or more than 3 times. Middle and high school students received two Think items (e.g., people 
deserve respect, even when they disagree with me; talking behind another person’s back is not that bad) 
rated as strongly agree, agree, undecided, disagree, strongly disagree; nine Do items (e.g., treat others the 
way I would like to be treated; treat people who are different from me with kindness and respect; listen 
to other students during class discussion) rated as strongly agree, agree, undecided, disagree, strongly 
disagree; and two How Many items (e.g., been mean to someone who hurt feelings; talked back to 
teacher or school staff) rated for frequency in the past year with 0 times, 1-2 times, 3-4 times, 5 times or 
more. 




X 


Competencies/skills: 

responsibility 


Teachers rated four items regarding students’ Responsibility (takes responsibility for choices; 
completes assignments; makes responsible choices in what say and do; forgets work supposed to do) on 
a 5-point Likert-type scale ranging from “strongly agree” to “strongly disagree.” 




9VV 



PROGRAM NAME: 



Level Construct/Components 

Teacher/Administration Level 

Professional efficacy 



Staff morale 



A. 16c 



FOR CHARACTER EDUCATION (ICE) 

;r Outcomes 

Measurement Information 



Teachers responded to Milson’s Character Education Efficacy Belief Instrument 
(CEEBI; Milson 2001), a 24-item measure addressing Personal Teacher Efficacy 
(12 items; e.g., “I am confident in my ability to be a good role model.” “I often 
find it difficult to persuade a student that respect for others is important.” and “I 
am continuously finding better ways to develop the character of my students.”) 
and General Teacher Efficacy (12 items; e.g., “Teachers are usually not 
responsible when a child becomes more courteous.” “Some students will not 
become more respectful even if they have had teachers who promote respect.”). 
Responses were made on a 5-point Likert-type scale. 

In focus groups, teachers responded to the following question: “Has teaching 
character education affected you personally? If so, please explain how.” 





TABLE A. 17a 



JUST COMMUNITIES (JC) 



Program name: 


Just Communities (JC) 


Program type: 


Comprehensive 


Grade level: 


9-12 (10-12 studied) 


Studies reviewed: 


Kuther, T.L., and A. Higgins-D’Alessandro. “Bridging the Gap Between Moral 
Reasoning and Adolescent Engagement in Risky Behavior.” Journal of Adolescence, 
vol. 23, no. 5, 2000, pp. 409-422. 

Power, F.C., A. Higgins, and L. Kohlberg. Lawrence Kohlberg’s Approach to Moral 
Education. New York, NY: Columbia University Press, 1989. 

Power, Clark, and Ann Marie R. Power. “A Raft of Hope: Democratic Education and the 
Challenge of Pluralism.” Journal of Moral Education, vol. 21, no. 3, 1992, pp. 193-206. 


Description: 


A comprehensive, high-school level intervention, the Just Community school is a 
democratic community with the goals of enhancing student moral reasoning and 
increasing their sense of community or attachment to the school through group discussion 
of moral and normative issues (Power et al. 1989). The JC school transmits the values of 
society by teaching justice or by assisting students to develop a stronger sense of fairness, 
including an understanding of the underlying purposes of laws and rules for building and 
maintaining trusting relationships and creating, critiquing, and improving the social order 
of the program itself. In a JC school, each student and faculty member has one vote and 
all have a stake in the school. Weekly core-group advisory meetings are conducted, in 
which groups of 10 to 12 students meet with a faculty core-group leader to exchange 
thoughts, discuss both school and personal matters, and form personal bonds. Weekly 
community meetings of all students and teachers are held to formulate and discuss the 
fairness and other moral aspects of school issues before the community, and to discuss 
and decide upon school issues and policies, including rules and sanctions about risky 
behaviors, such as alcohol and drug use. Student discussion is encouraged, as exposure to 
the logic of higher stages of reasoning, especially the next highest stage, promotes moral 
reasoning (Blatt and Kohlberg 1975; Kohlberg 1984; Power et al. 1989), and group 
discussion is the basis of building school norms and enhancing the bond of community 
(see Power et al. 1989, for operational definitions of these concepts). 
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TABLE A. 17b 



PROGRAM NAME: JUST COMMUNITIES (JC) 
Student Outcomes 



Affective 


Behavioral Cognitive 


Constructs/Components 


Measurement Information 


X 


X 


Attitudes/ emotions 

Risk behaviors: substance abuse, 
violence, sexual risk-taking 


Student self-renorted using Questionnaire on Engagement in Riskv Behavior; included 
items on four types of risky activities: antisocial behavior (theft, violence), substance 
involvement (alcohol, marijuana, illicit drugs and selling drugs), sexual involvement 
(engagement, unprotected) and suicidal thoughts. The items were phrased as follows: 
“Do you ? If so, how often?”; responses to the second half of the question 

addressing frequency occurred on seven point scale ranging from “never” to “almost 
every day.” 


X 


X 


Attitudes/motives: attitudes 
toward risk/health 

Reasoning: moral/ethical 
reasoning 


Student self-renorted on Percentions of Riskv Behavior using questionnaire on four 
types of risky activities: antisocial behavior (theft, violence), substance involvement 
(alcohol, marijuana, illicit drugs and selling drugs), sexual involvement (engagement, 
unprotected) and suicidal thoughts. For each item, participants were asked, “What is the 
basis of your decision of whether the act is right or wrong?” and were presented with 
three choices: (1) “It is right/wrong regardless of existing laws, rules, or social norms” 
(moral decision); (2) “It is right/wrong based on parental rules, laws, or social norms” 
(conventional decision); (3) “It is not right or wrong, but a matter of personal choice” 
(personal decision). 




X 


Reasoning: moral/ethical 
reasoning 


Students were directly assessed via an adapted Defining Issues Test (Rest 1986). 
Participants were presented with three short vignettes; after reading each vignette, 
participants rated the importance of 12 statements representing various stages of 
reasoning (preconventional, conventional, or postconventional). A postconventional 
reasoning score (indicating internalization of moral considerations) was derived from 
those ratings. Similar scores were derived for the percent of student reasoning at the 
preconventional and conventional levels (i.e., with higher percentage scores indicating a 
lesser degree of internalization). 




X 


Reasoning; moral/ethical 
reasoning 


Students were indirectly assessed using the Standard Moral Judgment Interview (Colbv 
et al. 1987) which asked students to respond to four vignettes concerning practical 
dilemmas that occur in high school. The vignettes included a helping dilemma, 
restitution dilemma, stealing dilemma, and drug dilemma. Responses were scored for: 
collectiveness of school norms; phase of development of school norms; stage of school 
norms; level of institutional valuing of school norms; stage of community in school; 
stage of individual moral reasoning about school norms; and judgments of personal 
responsibility. The scoring method corresponded to Kohlberg’s six levels of moral 
development (from externally enforced to internalized morality). 




TABLE A. 18a 



LEARNING FOR LIFE (LFL) 



Program name: 


Learning for Life (LFL) 


Program type: 


Modular 


Grade level: 


K-12 (2, 4, and 6 studied) 


Studies reviewed: 


Syndics Research Corporation and Kevin Ryan. “Character Building with Learning for 
Life.” Irving, TX: Learning for Life, n.d. 


Description: 


Learning for Life is a modular program, which provides a series of lessons to support 
students’ interpersonal skills, leadership, character development, cultural competency, 
and problem solving/critical thinking. The K-12 program has separate curricula by grade 
level (K-2, 3-4, 5-6, 7-8, 9-12). Lessons can be employed as stand-alone activities or may 
be integrated within ongoing curricula. Elementary curricula include 61 lesson plans and 
middle school features 44. Lessons bring together character development, career 
education, life skills (e.g., health, hobbies, money management, pet care), and academic 
learning. Topics include accepting consequences, not giving up, respect, gangs, and 
violence. Students participate in role plays and small groups, as well as completing 
activity sheets. Tasks involve reflection and discussion of moral dilemmas. 
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PROGRAM NAME: 



Affective Behavioral Cognitive Constructs/Components 

X X Knowledge: understanding 

values/norms; interpersonal 
knowledge 

Reasoning: moral/ethical reasoning 

Attitudes/motives: attitudes toward 
diversity 



X 



Prosocial behaviors 



A. 18b 




Measurement Information 

Student knowledge of specific content taught in the lessons was directly assessed 
using questionnaires (within a week of lessons) and surveys (pre/post). Item format 
varied by grade level. Instruments included a series of statements, for which students 
had to select true or false. Items covered cultural appreciation (e.g., “many different 
cultures have contributed to the success of America”), trust, prejudice, interpersonal 
skills, ethical/moral reasoning. 

Teacher reports on nine items regarding student behaviors (e.g., works well with 
others, is honest, cares about others, makes good choices). Items were rated on a 7- 
point scale from poor (1) to excellent (7). 




TABLE A. 19a 



LINKING THE INTERESTS OF FAMILIES AND TEACHERS (LIFT) 



Program name: 


Linking the Interests of Families and Teachers (LIFT) 


Program type: 


Modular 


Grade level: 


K-5 ( 1 and 5 studied) 


Studies reviewed: 


Reid, John B., J. Mark Eddy, Rebecca A. Fetrow, and Mike Stollmiller. ‘‘‘Description and 
Immediate Impacts of a Preventive Intervention for Conduct Problems.” American 
Journal of Community Psychology’, vol. 27, no. 4, 1999, pp. 483-517. 




Eddy, J. Mark, John B. Reid, and Rebecca A. Fetrow. “An Elementary School-Based 
Prevention Program Targeting Modifiable Antecedents of Youth Delinquency and 
Violence: Linking the Interests of Families and Teachers (LIFT).” Journal of Emotional 
and Behavioral Disorders, vol. 8, no. 3, 2000, pp. 165-176. 




Stollmiller, Mike, J. Mark Eddy, and John B. Reid. “Detecting and Describing Preventive 
Intervention Effects in a Universal School-Based Randomized Trial Targeting Delinquent 
and Violent Behavior.” Journal of Consulting and Clinical Psychology’, vol. 68, no. 2, 
2000, pp. 296-306. 


Description: 


Linking the Interests of Families and Teachers (LIFT) aims to target both child and parent 
behaviors across settings. As such, the LIFT program includes a classroom component, a 
school playground component, a parent component, and family involvement. The 
classroom component entails 1-hour lessons twice a week for 10 weeks on social skills 
and problem solving, specific to developmental needs of first versus fifth graders. For 
example, curriculum topics include listening, controlling anger, giving compliments, 
joining a group, and cooperating. Lessons include direct instruction and role play, 
practice time (small- and large-group), and skill review with daily rewards (class, small 
group, and individual). Games are used to encourage positive interactions and discourage 
negative behaviors. The parent component focuses on group training meetings (one a 
week across six weeks), with follow-up in person or by mail for missed meetings. Topics 
include discipline (appear calm, catch earlier, positive and negative consequences) and 
management (listening, effective requests, cooperation, encouragement). To promote 
family involvement, parents and teachers use a “LIFT-line” phone to leave messages for 
one another about home and school activities. Teachers also send home weekly 
newsletters and call parents. 
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TABLE A. 19b 



PROGRAM NAME: LINKING THE INTERESTS OF FAMILIES AND TEACHERS (LIFT) 

Student Outcomes 



Affective 


Behavioral Cognitive Constructs/Components 


Measurement Information 


X 


Competencies/skills: 
interpersonal competencies 

Risk behaviors 


The Interpersonal Process Code (Rusby, Estes, and Dishion 1991) was used to 
assess both maternal and child behaviors. To assess child physical aggression, 
children were observed on three separate occasions over a three week period for 
10 minutes during recess. Playground behavior was coded for content (verbal, 
nonverbal, physical behavior), and child physical aggression was identified as any 
act that fell in the “negative physical" content code (aversive behavior directed at 
another person, such as flicking, kicking or pinching, that is done with affect such as 
anger, displeasure, or harshness). 


X — 


Competencies/skills: 
interpersonal competency 


Teacher report on students’ social behavior using the Walker-McConnell Scale of 
Social Competence and School Adjustment Peer-Preferred Social Behavior subscale 
(Walker and McConnell 1995). Items included “share laughter,” “skillfully play 
games,” and “makes friends easily.” Neither the total number of items nor response 
options are specified. 
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TABLE A. 19c 



PROGRAM NAME: LINKING THE INTERESTS OF FAMILIES AND TEACHERS (LIFT) 

Other Outcomes 



Level 


C onstruct/ Components 


Measurement Information 


Parent/Community Level 




Parenting skills 


Maternal aversive behavior was coded using the Interpersonal Process Code (Rusby, 
Estes, and Dishion 1991). Parents and children completed a series of tasks (lasting 
30 minutes) in a videotaped laboratory visit. Tasks differed somewhat by age (first 
versus fifth grade). Tasks were then coded from videotapes by observers blind to 
intervention status for activity (i.e., setting), content (verbal, nonverbal, physical 
behavior), and affect (tone). Affect codes include: happy, caring, neutral, distress, 
aversive, and sad. Frequency, sequence and duration were also captured for each 
behavior. 






Coded behaviors included socially negative behaviors (“Negative Interpersonal” 
behaviors such as name calling or humiliation; “Noncompliance” behaviors such as 
ignoring requests) and socially neutral behaviors with aversive affect (i.e., display 
harshness, anger, displeasure). 





TABLE A. 20a 



LIONS QUEST— SKILLS FOR ACTION (LQ SKILLS FOR ACTION) 



Program name: 


Lions Quest — Skills for Action (LQ Skills for Action) 


Program type: 


Comprehensive 


Grade level: 


9-12 (9-12 studied) 


Studies reviewed: 


Laird, M., L.R. Bradley, and S. Black. “The Final Evaluation of Lions-Quest’s Skills for 
Action.” Newark, OH: Lions Quest, Lions Clubs International Foundation, 1998. 

Laird, M., and S. Black. “Service Learning Evaluation Project: Program Effects for At- 
Risk Students.” Unpublished manuscript, n.d. 


Description: 


Skills for Action-Lions Quest (SFA-LQ) is a comprehensive program to build positive 
character values, as well as life and citizenship skills for students in grades 9-12. The 
program includes classroom lessons and service learning. With more than 100 lessons 
focused around 26 personal, social, and thinking skills, SFA-LQ ranges from one 
semester to four years in length. Students explore personal stories, highlighting values 
and behavior through teachers’ questions, group discussion, and resource pages in the 
curricular materials. For service learning, students perform school-based or community- 
based projects with structured reflection on their experiences. Optional components 
include a student magazine, an Advisory Team, and supplemental units on substance use 
prevention. Lions Quest provides three different programs for different age groups: 
Skills for Growing (K-5), Skills for Adolescence (6-8), and Skills for Action (9-12). 
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TABLE A. 2 Ob 



PROGRAM NAME: LIONS QUEST— SKILLS FOR ACTION (LQ SKILLS FOR ACTION) 

Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X X 



X X 



X 



X 



Attitudes/motives: attitudes 
toward school, civic 
dispositions; internalizing 
problems 

Risk behaviors: substance use, 
discipline issues, 
absence/tardiness 

Prosocial behaviors: service, 
positive participation 



Attitudes/motives: civic 
dispositions, attitudes toward 
diversity 

Prosocial behaviors: service, 
positive participation 



Attitudes/motives: prosocial 
dispositions; attitudes toward 
school; intrapersonal strengths 



Risk behaviors: discipline 
problems 



Student self-report on two versions of the Life Review Survey (LRS), a 45-item questionnaire asking 
students about their lifestyle and risk behaviors. Version A contained all of the questions; version B was 
modified to omit questions deemed too sensitive by some administrators ( 1 1 items on suicide, 
depression, and drug use). The LRS includes three subscales: the Dropout Prediction Scale (12 items, 
e.g., “have any of your friends dropped out;” “do you expect to graduate”); the Alcohol and Drug Use 
Scale (9 items, e.g., “during the last three months, I have...” had alcohol, tried marijuana), and the 
Structured Time Use Scale (9 items, e.g., “during an average week, how many hours do you spend...” 
doing homework, watching television, in clubs/organizations). Response options varied by item content, 
including various scales for frequency of participation (never, once, twice, three or more times; no time, 
1-5 hours, 6-10 hours, etc.); true/false; and forced choice relating to content. A summative score was 
created across all items (ranging up to 80 points), where higher scores indicated lower risk. 

Student self report on the Service-Learning Survey , (Blyth et al. 1997) a 38-item instrument (adapted 
from the original 158-item instrument) designed to tap student attitudes and values about: involvement 
with diverse groups (e.g., “enjoy being around people whose background and experiences are 
different”); helping others (e.g., “believe that taking care of people who are having trouble taking care 
of themselves is everyone’s responsibility”); taking social action (e.g., “believe that most problems will 
solve themselves”); and intentions to volunteer in the community (e.g., “will be actively involved in 
political issues or social causes”). Response options varied by item content, including various scales for 
frequency of participation (no time, 1-5 hours, 6-10 hours, etc.); true/false; and Likert-type scales of 
agreement or likelihood. A summative score (ranging up to 115 points) was created across all items, 
with higher scores indicating a stronger service orientation. 

Student self report on the 35-item Checklist of Personal Gains (adapted from Conrad and Hedin 1980), 
gauged students’ perceptions of their own change in five areas: personal development, interpersonal 
development, values, attitudes toward academics, and career development. Students asked to respond 
“what is generally true for you right now,” with the post-test version adding “as a result of your 
service,” with responses on a 4-point Likert scale of agreement (l=strongly agree to 4=strongly 
disagree). Item examples include: “I am (more) able to get things done;” “I feel (more) responsible to 
the group or class;” “I have (more) realistic ideas about the world of work.” 

Administrative records of student suspensions over the course of the intervention were analyzed. 




TABLE A. 21a 



LIONS QUEST— SKILLS FOR ADOLESCENCE (LQ SKILLS FOR ADOL) 



Program name: 


Lions Quest — Skills for Adolescence (LQ Skills for Adol) 


Program type: 


Comprehensive 


Grade level: 


6-8 (6-9 studied) 


Studies reviewed: 


Eisen, M., G.L. Zellman, and D.M. Murray. “Evaluating the Lions-Quest ‘Skills for 
Adolescence’ Drug Education Program: Second-Year Behavior Outcomes.” Addictive 
Behaviors, vol. 28, no. 5, 2003, pp. 883-897. 


Description: 


Lions Quest - Skills for Adolescence is a school-wide program designed for middle 
school students (grades 6-8). The program was designed to promote good citizenship 
skills, core character values, and social-emotional skills and discourage the use of drugs, 
alcohol, and violence. The program includes a classroom curriculum, school-wide 
practices to create a positive school climate, parent and family involvement, and 
community involvement. The curriculum may vary in scope and intensity, lasting from 
nine weeks to three years. The lessons use cooperative group learning exercises and 
classroom management techniques to improve classroom climate. Lions Quest provides 
three different programs for different age groups: Skills for Growing (K-5), Skills for 
Adolescence (6-8), and Skills for Action (9-12). The schools in the current study used a 
condensed, 40-session version of the comprehensive, 103 -session version. 
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TABLE A.21b 



PROGRAM NAME: LIONS QUEST— SKILLS FOR ADOLESCENCE (LQ SKILLS FOR ADOL) 

Student Outcomes 



Affective Behavioral Cognitive Constructs/Components Measurement Information 



X 


X 


Attitudes/motives: attitudes 
toward risk/health 

Risk behaviors: substance use 


Student self-report on substance use with several items being adapted from the 
Monitoring the Future surveys (Johnston et al. 1996); substance use was 
collected for smoking (ever, past 30 days, and last 7 days; 3 items), alcohol, 
marijuana, cocaine, and other substances (ever, past 30 days; 8 items) on 6- or 
7-point scale of quantity (e.g., none to more than 100 cigarettes); intentions to 
use in next three months was collected separately for tobacco, alcohol, 
marijuana, and cocaine on 4-point scale (definitely yes to definitely no; 
4 items); perception of different peer group usage (best friend, friends, same- 
grade peers) collected separately on same substances (12 items) on 5-point 
scale (all to none). 


X 




Attitudes/motives: attitudes 
toward risk/health 


Student self report on Percentions of Substance Use with items adapted from 
the Monitoring the Future surveys (Johnston et al. 1996); three scales created 
on Peer Acceptance Via Substance Use (3 items) collected on alcohol, 
cigarettes, and drugs on 5-point response scale (strongly agree to strongly 
disagree); Effects of Substance Use (e.g., binge drinking, marijuana, smoking) 
whether help/harm health, relaxation, and popularity (15 items) on 4-point 
scale (very helpful to very harmful); Self-Efficacy to Resist was measured on 
different substances (alcohol, cigarettes, marijuana, cocaine) to say no at party, 
friend’s house, or hanging out (12 items) on 5-point scale (very hard to very 
easy). 




TABLE A.21c 



PROGRAM NAME: LIONS QUEST— SKILLS FOR ADOLESCENCE (LQ SKILLS FOR ADOL) 

Other Outcomes 



Level 


Construct/Components 


Measurement Information 


Parent/Community Level 




Parenting skills 
Participation in school 


Student report on Parental Monitoring scale using two items for whether parent 
knows student’s whereabouts or keeps track of how doing in school; responses on 5- 
point scale of frequency (never to always). 





TABLE A. 22a 



OPEN CIRCLE (OC) 



Program name: 


Open Circle (OC) 


Program type: 


Modular 


Grade level: 


K-5 (4 and 6 studied) 


Studies reviewed: 


Hennessey, Beth A. “Promoting Social Competence in School-Aged Children: The 
Effects of the Open Circle Program.” Journal of School Psychology, vol. 45, no. 3, 2007, 
pp. 349-360. 




Liang, Belle, Allison Tracy, Maureen Kenny, and Deirdre Brogan. “Gender Differences 
in the Relational Health of Youth Participating in a Social Competency Program.” 
Journal of Community Psychology, vol. 36, no. 4, 2008, pp. 499-514. 




Taylor, Catherine A., Belle Liang, Allison Tracy, Linda M. Williams, and Pamela Seigle. 
“Gender Differences in Middle School Adjustment, Physical Fighting, and Social Skills: 
Evaluation of a Social Competency Program.” Journal of Primary Prevention, vol. 23, 
no. 2, 2002, pp. 259-272. 


Description: 


Open Circle (OC) is a social emotional learning program, with a curriculum focusing 
instruction in social skills (e.g., problem solving, communication, conflict, cooperation, 
nonverbal signals, listening, dealing with teasing). The Open Circle Curriculum takes an 
ecological approach to learning and practicing new skills. Adults in children’s lives 
(teachers, administrators, caregivers) are trained as role models for communication, 
cooperation, and respect. The program begins in kindergarten and continues through 
elementary school (K-5). The curriculum includes numerous lessons delivered via regular 
circle times (twice a week). Open circle meeting times last about 15 to 30 minutes and are 
referred to as such because an empty chair is always included to demonstrate room for 
another person or voice. It is also noted as being a “context” to discuss issues, build self- 
esteem, and develop problem-solving skills. The developer notes the provision of 
materials for home newsletters, homework, and literature readings. 
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PROGRAM 



Affective Behavioral Cognitive Constriicts/Components 

Attitudes/motives: caring 

Competencies/skills: 
responsibility, intrapersonal 
competency, interpersonal 
X competency 



X X Attitudes/movies: internalizing 

problems 

Competencies/skills: 
intrapersonal competency; risk 
behaviors 

XXX Knowledge: academic content 

Attitudes/motives 

Competencies/skills 



A. 22b 



: OPEN CIRCLE (OC) 

Outcomes 

Measurement Information 

Parent, teacher, and student report using the Social Skills Rating System (SSRS; 
Gresham and Elliott 1990), with differing items across a variety of subscales. The 
items also vary across reporters to reflect the home versus classroom settings. All 
three report forms included subscales for Cooperation (items on behaviors like 
organization, focus, attention). Assertion (items on behaviors like speaking up for 
self and proactively interacting with others in a group), and Self-Control (items on 
behaviors like regulating behavior appropriately). These three subscales comprise 
the total Social Skills Scale for teachers (30 items; 10 Cooperation, 10 Assertion, 
10 Self-Control). 

The student form differs from the teacher report in that it includes an Empathy 
subscale (characteristics such as listening, demonstrating social overtures, 
understanding feelings) as part of the total Social Skills Scale. The student form has 
34 items total (10 Cooperation, 10 Assertion, 10 Self-Control, 10 Empathy; note that 
three items belong to more than one subscale). 

The parent form also differs in that it has a subscale for Responsibility (behaviors 
like request help, ask permission, provide information or question things). The form 
then includes 38 items total (10 cooperation, 10 assertion, 10 self-control, 
10 responsibility; one items belongs to two subscales). 

All reporters use a 3 -point scale of never, sometimes, or very often. 

Teacher report using Social Skills Rating System Problem Behavior Scale (SSRS; 
Gresham and Elliott 1990); 18 items across three subscales: Externalizing Problems 
like arguing or fighting, Internalizing Problems like depression or anxiety, and 
Hyperactivity like being fidgety or distractible; original manual notes a 3-point 
response scale of never, sometimes, and very often. 

Teacher report on academic achievement using the Social Skills Rating System 
Academic Competence Scale (SSRS; Gresham and Elliott 1990); 9 items on reading 
and math competence, motivation, behavior, and parent involvement; responses on 
5-point scale from “lowest 10%” to “highest 10%” on the child’s level relative to 
other children in the same classroom. 
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TABLE A. 22b ( continued ) 



Affective 


Behavioral Cognitive Constructs/Components 


Measurement Information 


X 


Attitudes/emotions 


Student report using the Relational Health Indices-Youth Version (Liang et al. under 
review); students identified a Close Friend, Mentor, and Community Group and 
were asked an unspecified number of items about their relationship with each (e.g., 
“I feel like this friend understands me,” “I can really be myself with this mentor,” 
and “Being a part of this group makes me feel good.). Questions regarding each 
entity (i.e., friend, mentor group) make up separate subscales. Responses ranged 
from “never” (1) to “always” (5). 


X 


Competencies/skills: 
interpersonal competency 

Risk behaviors: substance use, 
discipline issues 


Using a subset of items from the Survev of Adaptation Tasks-Middle School 
(Elias et al. 1992), students reported 19 items on three subscales including Peer 
Relations (treated like child, trouble making friends), Substance Use (alcohol, drugs, 
cigarettes), and Conflict (forget locker combination, tough teacher, sent to 
principal); responses on 4-point scale of “no problem” to “large problem.” 




Using one item from the Conflicts with Authority subscale, students reported 
whether or not they had a problem with fighting (response was either “yes” or “no”). 


X 


Attitudes/emotions: school 
bonding 


Parents and teachers reported on students’ adjustment to school. The Parent 
Report of School Adjustment included two items regarding how well their child is 
adjusting and how much they like school. Responses were made on a 4-point scale 
ranging from “not well/not much” to “very well/very much.” Teachers responded to 
one item on “overall adjustment” providing a rating of “poor,” “fair,” “excellent,” or 
“good.” 




TABLE A. 23a 



PARTNERSHIPS IN CHARACTER EDUCATION (PCE) 



Program name: 


Partnerships in Character Education (PCE) 


Program type: 


Comprehensive or modular 


Grade level: 


6-12 (6-12 studied) 


Studies reviewed: 


RMC Research Corporation. “Partnerships in Character Education: Evaluation Report.” 
Denver, CO: 2007. 


Description: 


Partnerships in Character Education is a program in the Philadelphia school district that 
provides professional development to teachers and classroom support for service-learning 
by connecting schools with community organizations through a coordinator or liaison. 
The organization Need in Deed focuses on connecting academics with the real world and 
provided teachers with 22 one-hour sessions throughout the year. Delaware Valley Earth 
Force provided three teacher workshops focused on environmental issues and projects. 
Educators were trained in a six-step model to engage students and provide them with real- 
world service activities. Champions of Caring provided four professional development 
sessions focused on transforming students into “champions” by supporting their efforts to 
complete service projects. Champions of Caring offers both student and teacher 
workshops and a curriculum called “Journey of a Champion” that addresses topics such 
as community, oppression, and self-reflection through service learning and character 
education. Through City Year, youth served as mentors or tutors and conducted 
community service projects. Schools developed partnerships with one or more of these or 
other organizations to achieve their character education goals. 
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PROGRAM NAME: 



Affective Behavioral Cognitive Constructs/Components 

XX X Attitudes/motives: civic dispositions; 

attitudes toward school 

Attitudes/emotions: school bonding 

Competencies/skills: respect, 
intrapersonal competency 

Prosocial behaviors: service, 
kindness, positive participation 

Reasoning 

X Knowledge: academic content 



XX X Attitudes/motives 

Competencies/skills: respect; 
interpersonal competency 

Prosocial behavior 

Knowledge: understanding 
values/norms 

Reasoning 

X Risk behaviors: absence/tardiness, 

discipline issues 



A.23b 



5 IN CHARACTER EDUCATION (PCE) 
Outcomes 



Measurement Information 

Student self-report using a Survey of Student Behavior and Affect ; the 10 subscales/topics 
included: Citizenship (2 items), Civic Engagement (3 items), Altruism (4 items), Social 
Responsibility (3 items), Caring (9 items), Respect (4 items), Ability to Choose between 
Right and Wrong (1 item), Efficacy (4 items), Persistence (2 items), Internal Locus of 
Control (2 items) and Value of School (10 items). 



Student academic achievement as indicated by Pennsylvania System of School 
Assessment (PSSA; Pennsylvania Department of Education), the state test used to comply 
with NCLB; standards based and criterion referenced. The PSSA tests students in reading 
and mathematics in grades 3-8 and grade 1 1 and writing in grades 5, 8, 11; rankings include 
advanced, proficient, basic, or below basic. 

Student achievement was also measured with the TerraNova (CTB/McGraw-Hill), to 
assess Language, Mathematics, Writing, and Science. The TerraNova offers both English 
and Spanish (the SUPERA) versions. 

Teachers responded to 15 items addressing student behaviors, attitudes, and 
knowledge on topics such as citizenship, efficacy, motivation, attendance, problem solving, 
and respect. Teachers rated perfonnance as either “low,” “moderate,” or “high.” 



Average daily attendance records were gathered from the Philadelphia school district. 

Administrative data gathered on discipline records regarding both in school and out of 
school suspensions. Records were collected from the district at the school level. 

School records were collected on the exact number of serious misconduct incidents 
reported. 




TABLE A. 24a 



POSITIVE ACTION (PA) 



Program name: 


Positive Action (PA) 


Program type: 


Comprehensive or modular 


Grade level: 


K-12 (1-5 studied) 


Studies reviewed: 


Flay, B., A. Acock, S. Vuchinich, and M. Beets. “Progress Report of the Randomized 
Trial of Positive Action in Hawaii: End of Third Year of Intervention." Twin Falls, ID: 
Positive Action, Inc., 2006. 


Description: 


Positive Action, a K-12 program, aims to promote character development, academic 
achievement, and social-emotional skills and to reduce disruptive and problem behavior. 
The program is based on the philosophy that you feel good about yourself when you think 
and do positive actions, and there is always a positive way to do everything. The 
curriculum includes six units; some grades have a review for a seventh unit. All lessons 
are scripted and use classroom discussion, role-play, games, songs, and activity sheets or 
text booklets. Optional components that may or may not be implemented as part of the 
program are: site-wide climate development; drug education for grade 5 and middle 
school; conflict resolution; counselor, parent, and family classes; and 
community/coalition components. 
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TABLE A.24b 



PROGRAM NAME: POSITIVE ACTION (PA) 
Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/Components 


Measurement Infonnation 




X 




C ompetencies/ skills 

Prosocial behaviors 

Risk behaviors: substance 
use 


Teachers completed a 75-item Student Behavior Checklist adapted from the Child Behavior Profile and the 
Teacher Child Rating Scale. iEdelbrock and Achenback 1984: Hightower et al. 1986). The 17 subscales 
address positive (4 items) and negative (4 items) aspects of self-concept, positive (3 items) and negative (2 
items) physical behaviors, positive (6 items) and negative (2 items) intellectual behaviors, responsibility (6 
items), self control (4 items), disruptiveness (6 items), consideration (7 items), respect (5 items), social 
behaviors (3 items), positive (6 items) and negative (5 items) aspects of honesty, self-improvement (6 
items), avoidance of substance use (3 items), and substance use (3 items). Neither individual items nor the 
response scale are provided. 


X 


X 




Attitudes/motives 
Prosocial behaviors 
Risk behaviors 


Students responded to survevs on their Involvement in and Interpretation of Own Behaviors: questions 
addressed their involvement in positive and negative behaviors and how they felt about themselves when 
engaging in those behaviors. Individual items are not presented. Up to grade 3, the response options include 
“no” (1), “sometimes” (2) and “yes” (3). At grade 4, response options changed to “none of the time” (1), 
“some of the time” (2), “most of the time” (3) and “all of the time” (4). Students also responded to five 
questions on Helping Behaviors such as “Have you helped someone who was hurt.” The response scale is 
not provided. 


X 






Attitudes/motives: 
attitudes toward school 

Attitudes/emotions : 
school bonding 


Students responded to survevs addressing Feelings Toward Others and the School with questions on how 
much they like others including peers and adults and the degree to which they feel attached to their school. 
The scale included five items when children were in early elementary school and ten items when they were 
in later elementary school. Factor analysis suggested three components for the scale: peers, adults, and 
school/teachers. Neither individual items nor the response scale are provided. 




X 




Risk behaviors: substance 
use and violence 


Once students reached fifth grade, they responded to substance use questions regarding tobacco, alcohol, 
marijuana, binge drinking, and having been drunk or high. Questions on violence addressed carrying a knife 
or razor to hurt someone, threatening to stab or cut someone, cutting or stabbing someone on purpose to 
hurt them, carrying a gun or shooting someone. For both substance use and violence, the authors created 
indicators of whether students had ever engaged in any of the behaviors. 


X 






Attitudes/motives 


In later elementary school, students responded to questions about their Attitudes and Intentions Regarding 
Positive Action. Six items addressed their attitudes toward the PA program, and three items addressed their 
intentions to use the positive actions they had learned in the program outside of school. Neither individual 
items nor the response scale are provided. 




X 




Risk behaviors: 
absence/tardiness 


Average daily absences. The source of the infonnation is not reported. 






X 


Knowledge: academic 
content 


Students knowledge of academic content was assessed with two types of information drawn from the 
standardized Hawaii Content and Performance Standards Test: (1) the percent proficient in grade 5 reading 
and (2) Adequate Yearly Progress scores calculated for No Child Left Behind requirements. 
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TABLE A.24c 



PROGRAM NAME: POSITIVE ACTION (PA) 
Other Outcomes 



Level 


Construct/ Components 


Measurement Information 


Teacher/Administration Level 




Understanding values/norms 
Staff morale 
Professional efficacy 


In this Survev of Teacher Attitudes, teachers reported their attitudes on a varietv of 
topics: teaching self-concept (5 items), self-efficacy to teach positive behaviors 
generally (14 items), school responsibility to teach social and character development 
(14 items), self-efficacy to teach specific positive behaviors (33 items), attitudes 
toward parents (5 items), attitudes toward students (10 items) attitudes toward other 
teachers (7 items), attitudes toward administrators (7 items), attitudes toward 
working in the school (8 items), attitudes towards Positive Action (4 items) and the 
degree of positive reinforcement of positive behaviors (4 items). Neither individual 
items nor the response scale are provided. 


School/Class Level 


Support 


School climate: interactions 


Students responded to a six-item Experiences of Harassment at School scale which 
introduced each question with the phrase, “How often do other kids at this school. . .” 
and included behaviors such as teasing and leaving the student out on purpose. The 
response scale is not reported. 




School climate: interactions; 
inclusion 


The Hawaii Department of Education School Quality Survev (SOS) asks students, 
parents, and teachers for their opinions of school quality and is administered every 
two years. This study used parent responses to a subscale on Support for Parent 
Involvement that addresses issues such as whether the school actively seeks to 
involve parents in helping achieve the schools’ goal and whether they are involved 
in planning. The exact number of items is not reported. Responses occur along a 5- 
point Likert-type scale ranging from “strongly disagree” (1) to “strongly agree” (5). 


Parent/Community Level 




Participation in school 


In later elementary school, students responded to questions about Family 
Involvement in Positive Action, a two-item scale. Neither individual items nor the 
response scale are provided. 




climate/environment 


In this Survev of Teacher Attitudes, teachers responded to three subscales regarding 
the neighborhood: Perception of Neighborhood Facilities (9 items), Neighborhood 
Sense of Community (6 items), and Social Disorganization of Neighborhood (4 
items). 







TABLE A. 25a 



PROJECT ESSENTIAL (PE) 



Program name: 


Project Essential (PE) 


Program type: 


Modular 


Grade level: 


Pre K-7 (K-8 studied) 


Studies reviewed: 


Dunn, L.S., and D.E. Wilson. “Moral Classrooms: The Development of Character and 
Integrity in the Elementary School.” Kansas City, MO: Teel Institute for the 
Development of Integrity and Ethical Behavior, n.d. 


Description: 


A modular classroom intervention. Project ESSENTIAL (PE) and its associated 
curriculum the ESSENTIAL Curriculum (EC) is designed for grades preK - 7 (additional 
grade levels are under development). EC focuses on teaching skills, behaviors, and 
attitudes necessary for every young person to lead a successful, secure, and productive 
life. For each grade level, the curriculum is divided into three types of learning and 
teaching: principles, supporting concepts, and skills. The scripted curriculum is organized 
conceptually around: (A) “Key Principles”: (1) admitting, correcting, and learning from 
mistakes; (2) identifying the appropriate roles of emotion and reason; (3) learning to 
identify and fulfill one’s true responsibilities; and (4) respecting one’s own rights and the 
rights of all others; (B) “Supporting concepts and curriculum topics”: (1) human 
individuality; (2) the proper ways to seek excitement and fun; (3) the roles of effort and 
pride; and (4) human relationships; and (C) “Skills”: (1) goal-setting; (2) moral 
reasoning; (3) introspection; and (4) empathy. The same topics are taught year after year 
with different, developmentally appropriate techniques and activities. Ideally, the 
curriculum is taught at least on a weekly basis. 
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TABLE A. 2 5b 



PROGRAM NAME: PROJECT ESSENTIAL (PE) 
Student Outcomes 



Affective 


Behavioral Cognitive 


Constructs/Components 


Measurement Information 


X 




Competencies/skills: responsibility; 
respect; interpersonal competency; 
intrapersonal competency 


Teachers rated students using a Behavioral Inventorv. In an unspecified 
number of items, ratings are made for four clusters of behaviors: 
(1) willingness to admit and correct mistakes; (2) respecting their own 
rights and the rights of others; (3) exhibiting self-discipline; and (4) 
reliability in meeting responsibilities; the study indicates that scores may 
range from 0-150, but the details of the scoring system are not provided. 


X 


X 


Attitudes/motives: intrapersonal 
strengths 

Competencies/skills: intrapersonal 
competency 


Students provided self-report using a self-esteem inventory. In an 
unspecified number of items, students addressed areas such as self-control, 
self-discipline, self-governance, and responsibility. Specific items are not 
provided. The response scale is not specified. The study does not report 
whether items were used as a scale. 




X 


Knowledge: understanding 
values/norms; academic content 


The Knowledge Test is a paper-and-pencil direct assessment of students’ 
knowledge regarding both general concepts as well as items specifically 
linked to the “ESSENTIAL principles” (e.g., responsibility, empathy, rights 
of others, emotions, goal-setting, and mistakes). No systematic information 
given on item content. 




TABLE A. 26a 



PROJECT HEART, HEAD, HANDS (H3) 



Program name: 


Project Heart, Head, Hands (H3) 


Program type: 


Modular 


Grade level: 


K-6 (2-6 studied) 


Studies reviewed: 


Furco, A., E. Middaugh, M. Goss, S. Darche, J. Hwang, and T. Tabemik. “Project Heart, 
Head, Hands: A Study of Character Development in Elementary School Students.” 
Hayward, CA: Alameda County Office of Education, 2004. 


Description: 


This project combines “proven” curricula and educational strategies into a modular 
system that is aligned and integrated with standards-based curricula of two of California’s 
language arts programs — SRA’s Open Courts and Houghton Mifflin Reading. The idea is 
that developing character education in concert with content standards is critical. Service- 
learning is linked to the K-6 curriculum. Teachers receive professional development to 
train them on the curriculum in the summer and meet monthly to discuss the progress. 
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TABLE A. 2 6b 



PROGRAM NAME: PROJECT HEART, HEAD, HANDS (H3) 
Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X X 



X 



X 



Attitudes/motives: attitudes toward 
diversity; intrapersonal strengths 

Competencies/skills: integrity; 
intrapersonal competency; 
responsibility; resistance 

Prosocial behaviors: service; kindness 



Student self-report using the 25-item Character Asset Questionnaire . Items are based 
on a subset of the 40 developmental assets identified by the Search Institute. The 
Integrity and Honesty subscale has four items addressing subjects such as telling the 
truth and waiting one’s turn. The Responsibility for Oneself subscale has four items 
addressing subjects such as paying attention, persistence, and trying one’s best. The 
Caring and Social Equity subscale has eight items addressing topics such as helping 
others, making the neighborhood a better place, and taking care of one’s school. The 
Social Competence subscale included five items on topics such as a positive outlook 
and making friends who are different from oneself. The Interpersonal 
Communication subscale has four items on topics such as not listening or interacting 
with others who tease or do bad things. Responses were made on a 4-point scale 
ranging from “Never” to “Always.” 



Reasoning 

Attitudes/motives 



Indirect assessment of students’ character development using a group- 
administered instrument. Students are asked to read four different scenarios with the 
researchers. Each scenario presents a dilemma and four possible action steps (e.g., 
Scenario: playground equipment that should have been brought in is left out; 
Possible action steps: (1) tell last person to go back, (2) go inside and not say 
anything, (3) pick up yourself, (4) tell last person and go with him/her to pick up. 
Students are asked to select the one action step that they would be most likely to 
take in response to the situation. Each of the four possible responses serves as an 
indicator for one or more character trait (e.g., caring, social equity, problem 
solving). Once they complete the scenarios, the researchers facilitate a discussion 
with the students, probing them for the reasons why they made the selections that 
they did. The researchers then document students’ verbal responses to capture the 
rationale behind the choices. Responses are then quantified. 




TABLE A. 27a 



RAISING HEALTHY CHILDREN (RHC) 



Program name: 


Raising Healthy Children (RHC) 


Program type: 


Comprehensive 


Grade level: 


K-12 (1-3 and 6-10 studied) 


Studies reviewed: 


Brown, Eric C., Richard F. Catalano, Charles B. Fleming, Kevin P. Haggerty, and Robert D. 
Abbott. “Adolescent Substance Use Outcomes in the Raising Healthy Children Project: A Two- 
Part Latent Growth Curve Analysis.” Journal of Consulting and Clinical Psychology, vol. 73, 
no. 4, 2005, pp. 699-710. 




Catalano, Richard F., James J. Mazza, Tracy W. Harachi, Robert D. Abbott, Kevin P. Haggerty, 
and Charles B. Fleming. “Raising Healthy Children Through Enhancing Social Development in 
Elementary School: Results After 1.5 Years.” Journal of School Psychology, vol. 41, no. 2, 
2003, pp. 143-164. 




Haggerty, Kevin P., Charles B. Fleming, Richard F. Catalano, Tracy W. Harachi, and Robert D. 
Abbott. “Raising Healthy Children: Examining the Impact of Promoting Healthy Driving 
Behavior Within a Social Development Intervention.” Prevention Science, vol. 7, no. 3, 2006, 
pp. 257-267. 




Harachi, Tracy W., Robert D. Abbott, Richard F. Catalano, Kevin P. Haggerty, and Charles B. 
Fleming. “Opening the Black Box: Using Process Evaluation Measures to Assess 
Implementation and Theory Building.” American Journal of Community Psychology, vol. 27, 
no. 5, 1999, pp. 711-731. 


Description: 


Raising Healthy Children aims to reduce teenage antisocial behavior and mental health 
problems. To achieve these goals, the program establishes teacher-parent relationships; 
enhances opportunities for positive interactions among teachers, parents, and students; and 
builds students’ positive social skills. 

Across elementary school and middle school, the teacher intervention includes school-level 
workshops on classroom management topics such as proactive management, cooperative 
learning, and problem solving skills. Each teacher also receives individual coaching. Parents 
work with a “school-home coordinator” in family management workshops and in school and in- 
home sessions. 

In early elementary school, students with academic or behavior problems participate in summer 
camps and experience in-home sessions with a school-home coordinator. As they get older, 
students volunteer in tutoring and study clubs. During middle and high school, students attend 
one-on-one sessions and group workshops. Social skill workshops are offered during middle 
school. Through all of these activities, students learn social skills and problem solving. 
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TABLE A. 2 7b 



PROGRAM NAME: RAISING HEALTHY CHILDREN (RHC) 
Student Outcomes 



Affective 


Behavioral Cognitive 


Constructs/Components 


Measurement Information 


X 




Attitudes/motives: attitudes 
toward school 


Commitment to School (Hawkins et al. 1992) was measured with both teacher and parent 
reports via two items from the Seattle Social Development Project (“tries hard in school" 
and “wants to do well in school”). Response options are unknown. 




X 


Knowledge: academic content 


Parents and teachers completed Academic Performance Rating for each student. 
Teachers compared students’ performance to their peers in reading, language arts, and 
math, and parents compared performance in reading and math. Response options are 
unknown. 


X 




Competencies/skills: 
interpersonal competency 


Students, teachers, and parents all rated student’s Interpersonal Competencv using 
separate scales. Students completed self reports on two yes/no items (“is it easy for you 
to make friends at school" and “is it easy for you to ask kids who you don’t know if you 
can join them in a game”). Parents completed seven items on behaviors such as making 
friends and working out conflicts. Teachers rated nine items drawn from the Teacher 
Observation of Classroom-Revised (Werthamer-Larsson et al. 1990) and the Walker- 
McConnell Scale of Social Competence (Walker and McConnell 1988) on understanding 
other’s feelings, cooperating with others, accepting responsibility, and sharing. Teachers 
and parents responded on either 3- or 5-point scales, but no further details are provided. 




X 


Risk behavior 


Students, parents, and teachers all rated students’ Antisocial Behavior. In self-reports, 
students indicated whether or not they engaged in eight behaviors (e.g., lying, breaking 
things, making fun of others). Parents and teachers responded to 10 items drawn from the 
Teacher Observation of Classroom Adaptation-Revised (Werthamer-Larsson et al. 1990) 
and the Aggressive Behavior scale of the Teacher Report Form (Achenbach 1991). This 
scale tapped behaviors such as breaking things, lying, and fighting within the past month. 
Items had a 3-point scale where l=rarely or never true and 3=often true. 




X 


Risk behavior: substance use 


Students completed self-reports on the frequency of substance use for alcohol, marijuana, 
and cigarettes using a 6-point scale (0=no use in previous year to 5=20 or more times in 
past month for alcohol and marijuana; 5=40 or more cigarettes per day). 




TABLE A. 28a 



RESOLVING CONFLICT CREATIVELY PROGRAM (RCCP) 



Program name: 


Resolving Conflict Creatively Program (RCCP) 


Program type: 


Comprehensive 


Grade level: 


K-12 (K-8 studied) 


Studies reviewed: 


Aber, J.L., J.L. Brown, and S.M. Jones. “Developmental Trajectories Toward Violence in 
Middle Childhood: Course, Demographic Differences, and Response to School-Based 
Intervention.” Developmental Psychology, vol. 39, no. 2, 2003, pp. 324-348. 




Aber, J.L., S.M. Jones, J.L. Brown, N. Chaudry, and F. Samples. “Resolving Conflict 
Creatively: Evaluating the Developmental Effects of a School-Based Violence Prevention 
Program in Neighborhood and Classroom Context.” Development and Psychopathology, 
vol. 10, no. 2, 1998, pp. 187-213. 


Description: 


The Resolving Conflict Creatively Program (RCCP) is a comprehensive violence 
prevention program, designed for grades K-12. Developed by the non-profit Educators 
for Social Responsibility, in collaboration with the New York City Board of Education, 
the program seeks to develop students’ social and emotional skills to deal positively with 
conflict and diversity, and to teach educators to create collaborative and non-violent 
classrooms and schools. Teachers administer the program through a curriculum, which 
includes conflict resolution, anger management, and intergroup relations. Topics 
addressed include active listening, assertiveness, expressing feelings, perspective taking, 
cooperation, negotiation, and being aware of biases or prejudice. The program relies on 
peer mediation and cooperative learning. 
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TABLE A. 2 8b 



PROGRAM NAME: RESOLVING CONFLICT CREATIVELY PROGRAM (RCCP) 

Student Outcomes 



Affective 


Behavioral 


Cognitive 


C onstructs/ Components 


Measurement Infonnation 




X 




Risk behaviors: violence; 
discipline issues 


Teacher Checklist of Child Aggressive Behavior ( Dodge and Coie 1987) used to report the frequency of 
six behavior items (e.g., “When this child is teased or threatened, he or she gets angry easily” and “This 
child threatens or bullies others in order to get his or her own way”). Responses given on Likert-type 
scale ranging from 1 (never) to 5 (always). 




X 




Prosocial behaviors 


Teacher report on child prosocial behavior using 19 items from the Social Competence Scale (Conduct 
Problems Prevention Research Group 1991), the Prosocial/Communication skills and Emotional 
Regulation skills subscales. Teachers reported how well the items (e.g.. the child “is helpful to others” 
and “acts friendly towards others”) described the child with response options ranging from “not at all” 
(1) to “very well” (5). 


X 


X 




Attitudes/ motives : 
internalizing problems; 
intrapersonal strength 

Risk behaviors: violence, 
discipline issues 


Students reported on adapted form of the Seattle Personalitv Inventorv. Depression and Conduct 
Problems subscales (Greenberg 1994). The Depression subscale used 1 1 items to assess depressive 
symptoms (e.g., "Do you feel unhappy a lot?” and “Do you feel that most things are not that much 
fun?”). The Conduct Problem subscale used 8 items (e.g., “Do you get into a lot of fights?” and “Do you 
tease or make fun of other kids?”). Response options for the study sample included yes, no, and don’t 
know. 


X 






Attitudes/ motives 


Six items from student reports on the “What I Think” instrument (Rosenfeld et al. 1982) were used to 
create a subscale for aggressive fantasies. Sample items included “Do you sometimes have daydreams 
about hitting or hurting someone you don’t like?” Response options were not reported. 


X 




X 


Attitudes/ motives 

Reasoning: moral/ethical 
reasoning 


The Home Interview (Dodge 1986) provided an indirect assessment of students’ hostile attribution bias 
and aggressive interpersonal negotiations. The Hostile Attribution subscale included six vignettes, which 
were read aloud while children viewed an illustration. For each vignette, children were asked to imagine 
themselves as having been provoked or offended by a peer, the cause of which was both visually and 
verbally ambiguous. The children were then asked about the cause of the provocation and asked to select 
one of four possible causal attributions, which were then scored as either hostile (1) or benign (0) and 
averaged across the six items. Following assessment of attributions of intent, children were asked what 
they would do next in each of the six scenarios, selecting from four possible response strategies. 
Forming the Aggressive Negotiation subscale, these responses were coded as either aggressive (1) or 
nonaggressive (0) and averaged across items. 






X 


Knowledge: interpersonal 
knowledge 


Indirect assessment via the Social Problem Solving Measure (Lochman and Dodge 1994). Researchers 
assessed children’s interpersonal skills through their responses to eight vignettes, which described 
problems typical of children’s social settings. Students’ solutions to the problems were coded as either 
competent (1) or noncompetent (0) and scores were averaged across items. 




TABLE A. 29a 



SOCIAL COMPETENCE PROMOTION PROGRAM FOR YOUNG ADOLESCENTS (SCPP-YA) 



Program name: 


Social Competence Promotion Program for Young Adolescents (SCPP-YA) 


Program type: 


Modular 


Grade level: 


6-9 (6-7 studied) 


Studies reviewed: 


Caplan, M., R.P. Weissberg, J.S. Grober, P.J. Sivo, K. Grady, and C. Jacoby. “Social 
Competence Promotion with Inner-City and Suburban Young Adolescents: Effects on 
Social Adjustment and Alcohol Use.” Journal of Consulting and Clinical Psychology 
vol. 60, no. 1, 1992, pp. 56-63. 




Weissberg, Roger P., H.A. Barton, and T.P. Shriver. “The Social Competence Promotion 
Program for Young Adolescents.” In Primary > Prevention Works, G.W. Albee and T.P. 
Gullotta. Thousand Oaks, CA.: Sage Publications, 1997. 


Description: 


The Social Competence Promotion Program for Young Adolescents (SCPP-YA) is a 
modular school prevention program that teaches students cognitive, behavioral, and 
affective skills and encourages them to apply these skills in dealing with daily challenges, 
problems, and decisions. The program targets African-American and white youths in 
middle and junior high schools. 




The 45-session SCPP-YA has three modules. The first module includes 27 lessons of 
intensive instruction in social problem-solving (SPS) skills. Students are taught to 
(1) stop, calm down, and think before they act; (2) express the problem (aloud) and how 
they feel; (3) set a positive goal; (4) think of lots of solutions; (5) think ahead to the 
consequences; and (6) go ahead and try the best plan. These foundational lessons are 
followed by two nine-session programs that teach students to apply SPS skills to the 
prevention of substance abuse and high-risk sexual behavior. To foster the application 
and generalization of SPS concepts and skills to daily life, teachers are trained to model 
problem-solving to students in situations other than formal classroom lessons and to 
guide and encourage students to try out problem-solving strategies in school, at home, 
and in the community. 
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TABLE A. 2 9b 



PROGRAM NAME: SOCIAL COMPETENCE PROMOTION PROGRAM FOR YOUNG ADOLESCENTS (SCPP-YA) 

Student Outcomes 



Affective Behavioral Cognitive Constructs/Components 

XXX Attitudes/motives: attitudes toward 

risk/health; intrapersonal strengths 

Competencies/skills: resistance; coping 

Reasoning: moral/ethical reasoning 



Competencies/skills: intrapersonal 
competency; interpersonal competency 



7X Attitudes/motives: internalizing 

problems 



X Attitudes/motives: prosocial 

dispositions; intrapersonal strengths 



Measurement Information 



To address Coping Skills , students were assessed in two ways. In an assessment 
adapted from Decision-Making Questionnaire (Gersick et al. 1988), students 
responded to hypothetical vignettes by listing procedures they would try in the face 
of peer pressure to smoke. In the second assessment, students listed the different 
things they would do to calm themselves down when feeling anxious or stressed. 
Neither individual items nor the total number of items are reported. For both 
assessments, responses were coded by outside raters in a two-step process in which 
non-redundant responses were counted and then qualitatively rated on a scale 
ranging from “not very effective” (1) to “very effective” (4) for the smoking 
vignettes and “very maladaptive” (1) to “very adaptive” (4) in the case of strategies 
for calming down. 

Teachers indicated how well four items described a student’s social and emotional 
adjustment (constructive conflict resolution with peers, impulse control, popularity, 
assertiveness with adults) on a scale ranging from “not at all” (1) to “very well” (5) 
(Allen et al. 1989). 

Using the Rand Well-Being Scale (Veit and Ware 1983), students responded to 
12 items assessing their general mood and emotional state that asked them to judge 
how often they experienced feelings such as loneliness, restlessness, and sadness. 
Responses were made on a 5-point scale. 

Students responded to two subscales of the 36-item Self-Perception Profile for 
Children (Harter 1985). The Behavioral Conduct subscale measures the degree to 
which students like the way they behave, act the way they are supposed to, and 
avoid getting into trouble. The Self-Worth subscale focuses on how much children 
like themselves. Individual items, the total number of items in the subscales, and 
response categories are not provided. 



X Attitudes/motives; intrapersonal strength In the Decision-Making Confidence Scale (Wills 1986), students rated how strongly 

they agreed with four statements (out of eight in the original scale) regarding 
decision-making using a 5-point scale. No additional information is provided. 
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TABLE A. 29b ( continued ) 



Affective Behavioral Cognitive Constructs/Components 



Measurement Information 



X 



X 



X 



X 



Attitudes/motives 



Attitudes/motives: attitudes towards 
risk/health 



Attitudes/motives: attitudes towards 
risk/health 



Risk behaviors: substance use 



In a measure of Problem Solving Efficacy , students indicated the usefulness of each 
of four problem-solving steps on a 4-point scale ranging from “not at all" (1) to 
“very useful" (4). Individual items are not reported. 

To measure students’ intent to use substances, they indicated what they would say if 
a friend offered them cigarettes, beer, wine, hard liquor, marijuana, cocaine, crack, 
depressants, or stimulants. Responses were made on a 5-point scale ranging from 
“definitely no” (1) to “definitely yes” (5). 

Two separate 10-item Attitudes Towards Substance Use (Botvin et al. 1984) scales 
assessed students’ attitudes toward smoking and drinking. Students responded to 
statements such as “Kids who smoke have more friends,” and “Drinking makes you 
look cool,” on a 5-point scale indicating the degree of agreement. 

Students were asked to indicate how often they had used cigarettes, beer, wine, 
hard liquor, marijuana, cocaine, crack, depressants, or stimulants in the past two 
months, with response options of never, less than once a month, once or twice a 
month, once a week, two or more times a week. An additional item with a fake drug 
(“donovites”) was included to gauge over-reporting. To explore excessive use, 
students responded to four questions about the frequency of drinking alcohol. No 
additional information is provided. Questions were adapted from Grady et al. 1986 
and Kandel et al. 1978. 




TABLE A. 30a 





SOCIAL DECISION MAKING/PROBLEM SOLVING (SDM/PS) 


Program name: 


Social Decision Making/Problem Solving (SDM/PS) 


Program type: 


Modular 


Grade level: 


K-8 


Studies reviewed: 


Elias, Maurice J., Michael A. Gara, Thomas F. Schuyler, Leslie R. Branden-Muller, and 
Michael A. Sayette. “The Promotion of Social Competence: Longitudinal Study of a 
Preventive School-Based Program.” American Journal of Orthopsychiatry’, vol. 61, no. 3, 
1991, pp. 409-417. 

Bruene-Butler, Linda, June Hamson, Maurice J. Elias, John F. Clabby, Jr., and Thomas F. 
Schuyler. “The Improving Social Awareness, Social Problem-Solving Project.” In 
Primary’ Prevention Works, G.W. Albee and T.P. Gullotta. Thousand Oaks, CA.: Sage 
Publications, 1997. 


Description: 


Social Decision Making/Problem Solving (SDM/PS) aims to be a primary prevention 
program by promoting social competence, in particular the ability to choose correct 
behaviors. Focal areas include decision making/problem solving, self-control, group 
participation, peer acceptance, and social awareness to apply skills, especially under 
stress. Elias et al. (1991) note this as prevention program during grade 4 and 5. Bruene- 
Butler et al. (1997), however, describe the program as serving both elementary and 
middle school students (K-6). The SDM/PS curriculum has three stages. First, the 
readiness stage focuses on learning skills and cues, with lessons on self-control (listening, 
turn taking, following multi-step directions), group participation, and social awareness 
(share ideas, ask/give/receive help, give/receive praise, choose friends). Second, the 
instructional stage focuses on an 8-step thinking process for decision making: (1) look for 
signs of different feelings, (2) identify problem, (3) set goal, (4) think of solutions, 
(5) reason each solution consequence, (6) choose best solution, (7) plan and check, (8) try 
and rethink. The 8-step program is spread across 22 lessons. Third, the application stage 
entails practice across settings (creative writing, social studies, starting projects) with 
reinforcement by teachers. The curriculum includes specifically ordered lessons with 
activities. Support for teachers is available as well as materials for including school- and 
family-level components in the intervention. 
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PROGRAM NAME: SOCIAL 



Affective Behavioral Cognitive 



C onstructs/ Components 



X X Attitudes/motives: internalizing 

problems 

Competencies/skills: 
interpersonal competency 

Risk behaviors 



X 



Attitudes/motives: intrapersonal 
strengths 



A. 3 Ob 




Measurement Information 

Items from the social and activities competence and clinical scales of the Y outh Self Report 
(YSR; Achenbach et al. 1987) were used to assess respondents’ social competence and 
psychopathology. The YSR also includes open-ended responses to items covering physical 
problems, concerns, and strengths. Youths rate themselves for how true each item is now or 
was within the past six months, using a 3-point response scale, ranging from “not true” (0) to 
“very true” or “often true” (2). Social Competence items gauged the quality of youths’ 
relationships (e.g., how they get along with parents or peers). Activity items captured 
information on number of clubs/organizations youth belong to, as well as how well they 
carry out job tasks or chores. The original instrument includes six clinical scales: depression, 
unpopularity, aggression, delinquency, somatic complaints, and thought disorder (with 
varying items for girls and boys; and an additional scale on self-destruction/identity 
problems for boys). The Elias et al. study included all items in the YSR, but then 
discriminant analysis ultimately led to the use of depression, delinquency, unpopularity, and 
self-destructive (males only) as individual scales. Researchers also dichotomized scale 
scores, using the YSR clinical cut-offs, to create a count of clinical problems (ranging 0 to 
7). 

Student self reports on the Perceived Competence Scale for Children (PCSC; Harter 1982) 
used 28 items to measure self-efficacy. The instrument includes a subscale for General Self- 
Worth, as well as multiple items in the domains of Cognitive, Social, and Physical 
Competency. Items for Self-Worth include, for example, “sure of myself,” and “happy the 
way I am” (7 items); for Cognitive Competence, “good at school work” and “can figure out 
answers” (7 items); for Social Competence, “have a lot of friends” and “easy to make 
friends” (7 items); for Physical Competence, “do well at all sports” and “play rather than 
watch” (7 items). Self Worth items are presented with a “structure alternative format” 
(designed to limit social desirability bias) in which children respond “really true for me” or 
“sort of true for me” on one of two paired alternative statements. The student first selects the 
option most like him/her (e.g., “some kids often forget what they learn but other kids can 
remember things easily”) then selects whether it is “really” or “sort of’ true. In the original 
PCSC, low vs. high competence options were varied on left and right, and scores ranged 
from 1 (low perceived competence) to 4 (high perceived competence). Elias et al. (1991) 
report the range as 4 to 112, indicating summing across items. Total scores were then 
divided into four levels: negative; mixed, mostly negative; mixed, mostly positive; positive 
(no information was provided on cut-points for these). 
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TABLE A. 30b ( continued ) 



Affective Behavioral 


Cognitive 


C onstructs/ Components 


Measurement Information 


X 




Risk behaviors: substance use, 
violence 


Student self reports on six items from the National Youth Survev (Elliot et al. 19831 were 
included to assess risk behavior: (1) use of cigarettes/tobacco; (2) buy/provide alcohol for 
others; (3) vandalism against parental property; (4) hit or threaten other student; (5) hit or 
threaten parents; and (6) attack another person. Students reported the frequency with which 
they had engaged in the behaviors over the past year, using a 7-point scale (0=never, 3=once 
every 2-3 weeks, 6=2-3 times a day). 




X 


Knowledge: academic content 


Direct assessment of student knowledge via the Comprehensive Test of Basic Skills (CTBS: 
CTB McGraw Hill). The CTBS is a norm-referenced, comprehensive, standardized 
achievement test covering multiple domains. Elias et al. (1991) report the composite score as 
well as sub-scores for language arts, math, and science (for 1 1th graders). 


X 




Prosocial behavior: positive 

participation 


Administrative records provided a count of student absences over the course of one school 
year. 




X 


Reasoning: critical 
thinking/decision making 


Indirect assessment of students’ interpersonal sensitivity, planning, and problem analysis 
with the Group Social Problem Solving Assessment (Elias et al. 1986). Students are 
presented with two vignettes, each containing a problem or dilemma (e.g., “You are in class, 
having a math lesson. The teacher reads from the book and calls on you to answer. You 
answer, but you are wrong. A few other children laugh at you. You are upset about this, and 
you do not want to be laughed at.”). The student then writes answers for a series of open- 
ended questions addressing what he or she thinks or should consider about the situation. 
These questions introduce contingencies that prevent students from posing obvious 
solutions. Scores may be calculated for consequences, planning, expectancies, alternative 
solutions, and problem-solving strategies. 




TABLE A. 31a 



TEEN OUTREACH 



Program name: 


T een Outreach 


Program type: 


Modular 


Grade level: 


7-12 (9-12 studied) 


Studies reviewed: 


Allen, J.S., S. Philliber, S. Herrling, and G.P. Kupermine. “Preventing Teen Pregnancy 
and Academic Failure: Experimental Evaluation of a Developmentally Based Approach.” 
Child Development, vol. 64, 1997, pp. 729-742. 

Allen, J.S., and S. Philliber. “Who Benefits Most from a Broadly Targeted Prevention 
Program? Differential Efficacy Across Populations in the Teen Outreach Program.” 
Journal of Community Psychology, vol. 29, no. 6, 2001, pp. 637-655. 

Allen, J.P, S. Philliber, and N. Hoggson. “School-Based Prevention of Teenage 
Pregnancy and School Dropout; Process Evaluation of the National Replication of the 
Teen Outreach Program.” American Journal of Community Psychology, vol. 18, no. 4, 
1990, pp. 505-524. 

Allen, J.P., G. Kupermine, S. Philliber, and K. Herre. “Programmatic Prevention of 
Adolescent Problem Behaviors: the Role of Autonomy, Relatedness, and Volunteer 
Service in the Teen Outreach Program.” American Journal of Community Psychology’, 
vol. 22, no. 5, 1994, 617-638. 


Description: 


The Teen Outreach Program is designed to prevent adolescent problem behaviors by 
enhancing normative processes of social development. The program seeks to engage 
young people in a high level of structured, volunteer community service that is closely 
linked to classroom-based discussions of future life options, such as those surrounding 
future career and relationship decisions. Students meet weekly to discuss their 
“developmental” challenges like life skills, dealing with family stress, human 
development, teen pressure, etc. Students also work in community service. 

In total, the program consists of three interrelated elements: supervised community 
volunteer service, classroom-based discussions of service experiences and classroom- 
based discussion and activities related to key social-developmental tasks of adolescence. 
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TABLE A. 3 lb 



PROGRAM NAME: TEEN OUTREACH 
Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/ Components 


Measurement Information 




X 




Risk behaviors: sexual risk taking 


Student self report on a single item if ever pregnant/responsible for a 
pregnancy. 




X 




Risk behaviors: discipline issues 


Student self report on a single item on whether suspended in prior year. 






X 


Knowledge: academic content 


Student self report on a single item on whether failed a course in prior year. 




TABLE A. 32a 



TOO GOOD FOR DRUGS & VIOLENCE (TGFDV) 



Program name: 


Too Good for Drugs & Violence (TGFDV) 


Program type: 


Modular 


Grade level: 


9-12(9-12) 


Studies reviewed: 


Bacon, T.P. “Impact on High School Students’ Behaviors and Protective Factors: A Pilot 
Study of the Too Good for Drugs and Violence Prevention Program.” Florida Educational 
Research Council, Inc., Research Bulletin, vol. 32, nos. 3 & 4, 2001, 

pp. 1-40. 


Description: 


TGFDV is designed to promote high school students’ prosocial skills, positive character 
traits, and violence- and drug-free norms. The curriculum consists of 14 core lessons 
(60 minutes each) and an additional 12 lessons that can be infused into other subject areas 
(such as English, science, and social studies). All lessons are scripted and intended to be 
taught by trained teachers or Too Good instructors. The program emphasizes prosocial 
skills, respect for others, and personal and social responsibility. Students engage in role- 
play and cooperative learning activities and are encouraged to apply the skills to different 
contexts. The program includes optional family and community involvement. (Related 
program, Too Good for Violence (K-8), also reviewed here.) 
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A. 32b 



OR DRUGS & VIOLENCE (TGFDV) 

Outcomes 

Measurement Information 

Student self-report using the four-item Intentions Scale (Bacon 2000). Items 
addressed intentions to use tobacco, alcohol, marijuana, or violence in next 
12 months. Responses were made on a five-point scale ranging from 
“strongly disagree” to “strongly agree.” 

Student self-report using the 61 -item Protective Factor Perceptions Survey 
(Bacon 2000). The nine subscales included Positive Attitudes toward Non- 
Drug Use, Positive Attitudes Toward Non-Violence, Perceptions of Peer 
Normative Substance and Violence Use, Perceptions of Peer Disapproval of 
Substance and Violence Use, Perceptions of Emotional Competence, 
Perceptions of Goal Setting and Decision Making Skills, Perceptions of 
Social and Peer Resistance Skills, Perceptions of Harmful Effects of 
Substance Use, and Perceptions of Parental Attitudes Toward Substance 
Use. Responses were made on a five-point scale ranging from “strongly 
disagree” to “strongly agree.” 




TABLE A. 33a 



TOO GOOD FOR VIOLENCE (TGFV) 



Program name: 


Too Good For Violence (TGFV) 


Program type: 


Modular 


Grade level: 


K-8 (3 studied) 


Studies reviewed: 


Bacon, T.P. “Impact on High School Students’ Behaviors and Protective Factors: A Pilot 
Study of the Too Good for Drugs and Violence Prevention Program.” Florida Educational 
Research Council, Inc., Research Bulletin, vol. 32, nos. 3 & 4, 2001, 

pp. 1-40. 




Hall, B.W., and T.P. Bacon. “Building a Foundation Against Violence: Impact of a 
School-Based Prevention Program on Elementary Students.” Journal of School Violence, 
vol. 4, no. 4, 2005, pp. 63-83. 


Description: 


A school-based “social influence intervention” with 7 units (30-60 min each). This K-8 
violence prevention/character education program seeks to improve student behavior and 
minimize aggression. TGFV teaches students positive attitudes, beliefs, and behaviors; 
builds skills sequentially with developmentally appropriate curricula designed to address 
the most significant risk and protective factors for each grade. 




TGFV promotes a CAREing approach to violence prevention by teaching: Conflict 
resolution; Anger management; Respect for self and others; and Effective 
communication. 




As designed, a trained teacher, counselor, or prevention specialist delivers TGFV in a 
classroom setting. Each grade-level kit includes: scripted curriculum, workbooks, and 
materials (e.g., posters, games, CDs, visual aids). Also includes Information and 
Exercises for Parents and Kids, to be copied and sent home. Highly interactive teaching 
methods encourage students to bond with prosocial peers, and engage students through 
role-playing, cooperative learning, games, small-group activities, and class discussions. 
TGFV teaches that each student has what it takes to solve conflicts peaceably and 
provides opportunities to practice peacemaking and antibullying skills. 
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TABLE A. 




Affective Behavioral Cognitive Constructs/Components 

XXX Competencies/skills: resistance; 

intrapersonal competency; interpersonal 
competency; communication 

Knowledge: interpersonal knowledge; 
intrapersonal knowledge 

Risk behaviors 



X Competencies/skills: respect; 

interpersonal competency 



X 



Prosocial behaviors 
Risk behaviors 

Knowledge: intrapersonal knowledge 



(TGFV) 



Measurement Information 

The Student Protective Factors Survey is a 32-item instrument gathering student 
self-reports on their Emotional Competency Skills (e.g., “I can calm myself down 
when I am upset,” and “I know many different words to describe what I feel 
inside.”), their Social and Conflict Resolution and Resistance skills (e.g., “If a 
student was bothering me, I would walk away,” and “I use peaceful ways to work 
out conflicts with other students.”), Communications Skills (e.g., “I can tell how 
students feel by listening to their tone of voice,” and “I use ‘I feel’ messages to share 
my feelings with other students.”) and Interactions with Other Students such as 
yelling and pushing. Students responded on a 5-point Likert-type scale ranging from 
“strongly disagree” (1) to “strongly agree” (5). The number of items in each 
subscale is not reported. 

The Teacher Checklist of Student Behaviors includes 21 -items addressing Personal 
and Social Skills (e.g., treats other students with respect, uses a variety of verbal 
labels for emotions), Positive Social Behaviors (e.g., helps other students; takes 
turns, plays, follows rules), and Inappropriate Behaviors (e.g., name calling, yelling, 
pushing). Teachers responded on a 5-point Likert-type scale ranging from “never” 
(1) to “almost always” (5). The number of items addressing each topic is not 
reported. 




TABLE A. 34a 



TOPEKA CHARACTER EDUCATION INITIATIVE/CHARACTER FIRST® 



Program name: 


Topeka Character Education Initiative/Character First 1 ' 


Program type: 


Comprehensive 


Grade level: 


K-12 (4, 7, 9 studied) 


Studies reviewed: 


Tatarko, Beth. “Topeka Character Education Initiative: Evaluation Report Prepared for 
the U.S. Department of Education.” Overland Park, KS: The Austin Peters Group, Inc., 
2007. 


Description: 


The Topeka Character Education Initiative (TCEI) took a multi-faceted approach to 
character education in Topeka Public Schools. First, Topeka City of Character, a 
nonprofit community organization that promotes good character throughout the city, 
partnered with the schools by providing resources and training to families and to conduct 
a community-wide media campaign. 




Second, Topeka Public Schools implemented character education activities in the schools 
through Character First, a nonprofit organization in Oklahoma City. Character First offers 
training seminars for teachers on how to implement and integrate character education 
throughout a school at all grade levels. In their three-step approach, teachers 
(1) emphasize character through classroom lessons, school assemblies, daily application, 
and volunteer mentoring; (2) require character by setting examples and taking advantage 
of teachable moments; and (3) recognize when good character is displayed. Teacher 
seminars include topics such as changing the culture, how to teach character, building 
character with praise, character-based correction of behavior, and steps for 
implementation. Character First emphasizes 49 different character qualities. It offers four 
curricula, each with nine character traits that can be implemented on a monthly basis 
throughout the school year. Teacher guides, DVDs, video and audio tapes, character 
cards, songbooks, and posters are available for each of the four sets of character qualities. 
Separate materials are available for elementary and secondary students. 




Finally, Topeka Public Schools formed a partnership with Professional Mentoring to 
sustain professional development of staff through online courses with a curriculum 
written by a member of the City of Character Council. 
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TABLE A. 34b 



PROGRAM NAME: TOPEKA CHARACTER EDUCATION INITIATIVE/CHARACTER FIRST® 

Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/ Components 


Measurement Information 


X 


X 




Attitudes/motives: attitudes toward 
school 

Attitudes/emotions: school bonding 

Competencies/skills: responsibility, 
intrapersonal competency 


Student self-renort and narent report on school experiences using Topeka 
Character Education Survey; 3 scales on Academic Abilities 15 items: being a 
good student, working hard, school easy), Work Ethic (3-4 items; homework 
completion, doing best is important), and Quality of School Life (5 items; like 
school, feel safe, treated fairly); responses on 5-point scale of agreement 
(strongly disagree to strongly agree). 


X 


X 




Attitudes/motives: prosocial disposition, 
intrapersonal strengths 

Competencies/skills: responsibility, 
respect, integrity 

Prosocial behavior: trustworthiness, 
service, positive participation 


Student self-report and parent report on attitudes and behaviors using Topeka 
Character Education Survey; 4 scales on Student Generosity (5 items; helping 
others, doing good deeds), Responsibility (3 items; take care of things, treat 
others with respect), Self-Confidence (4 items; class participation, feel good 
about self), and Trust/Truthfulness (5 items; trusted by others, tell truth); 
responses on 5-point scale of agreement (strongly disagree to strongly agree). 






X 


Knowledge: academic content 


Administrative data on academic achievement — reading test scores (instrument 
not specified), Kansas state assessment, pass/ fail rates, and grades. 








Prosocial behavior: positive participation 


Administrative data on attendance and graduation rates. 


X 






Attitudes/motives: in general, attitudes 
toward school 


Administrative data on post-high school goals (writing goals, career guidance, 
post-secondary enrollment, four-year academic plans). 


X 


X 




Risk behaviors: discipline issues 


Administrative data on suspensions. 




A.89 



TABLE A. 34c 



PROGRAM NAME: TOPEKA CHARACTER EDUCATION INITIATIVE/CHARACTER FIRST® 

Other Outcomes 



Level 


Construct/Components 


Measurement Information 


School/Class Level 




School climate 


Teacher report on student school experiences (across all students) using Topeka 
Character Education Survey; 3 scales on Academic Abilities 15 items; being a 
good student, working hard, school easy), Work Ethic (3 items; homework 
completion, doing best is important), and Quality of School Life (5 items; like 
school, feel safe, treated fairly); responses on 5-point scale of agreement 
(strongly disagree to strongly agree). 




School climate: collective norms/values, 
interactions 


Teacher report on student attitudes and behaviors (across all students) using 
Topeka Character Education Survev; 4 scales on Student Generosity (5 items; 
helping others, doing good deeds). Responsibility (3 items; take care of things, 
treat others with respect), Self-Confidence (4 items; class participation, feel 
good about self), and Trust/Truthfulness (5 items; trusted by others, tell truth); 
responses on 5-point scale of agreement (strongly disagree to strongly agree). 


Parent/Community Level 




Participation in school 


Student, teacher, and parent report on scale for Parent Participation in 
Schooling using Topeka Character Education Survev; 6 items such as talking to 
teacher/student on schoolwork and attending parent-teacher conferences; one 
additional item for students only on whether parents could name close friends; 
teacher ratings referred to students in general; responses on 5-point scale of 
agreement (strongly disagree to strongly agree). 






TABLE A. 35a 



TRIBES TLC 



Program name: 


Tribes TLC 


Program type: 


Comprehensive 


Grade level: 


K-12 (4-5 students, K-8 teachers studied) 


Studies reviewed: 


Chesswas, R.J.D., L.J. Davis, and T. Hanson. “Evaluation of the Implementation and 
Impact of Tribes TLC.” Windsor, CA: CenterSource Systems, LLC, July 2003. 

Spring Branch Independent School District. “PCEP Grantee Report on Project HEART: 
Implementation Report.” Spring Branch, TX: 2003. 


Description: 


Tribes TLC is a comprehensive, K-12 character education program centered on the 
development of learning communities within classrooms. Tribes teachers organize their 
students into collaborative learning groups of three to six students, with each "tribe” 
working together to promote a spirit of cooperation and social acceptance. At the 
program’s core, students and teachers agree to honor four basic agreements while in the 
classroom: (1) they agree to listen attentively to one another, (2) they promise to show 
appreciation for one another and avoid “put downs,” (3) they promise to show mutual 
respect, and (4) they agree that all students have the "right to pass” on peer-led activities 
in which they would rather not participate. 
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TABLE A. 3 5b 



PROGRAM NAME: TRIBES TLC 
Student Outcomes 



Affective 


Behavioral 


Cognitive 


Constructs/ Components 


Measurement Information 


X 


X 




Attitudes/motives: attitudes toward 
school 

Competencies/skills: respect, 
leadership 

Prosocial behaviors 

Risk behaviors 


Student self-report on feelings about and experiences at school; 20 items on 
activities in class (e.g., take turns speaking, put other people down, share 
how feel) with responses on 4-point scale of frequency ranging from “Yes, 
all of the time” to “No, never”; 1 1 items on attitude toward school (e.g., feel 
happy at school, help make class rules) with responses on 4-point scale of 
frequency ranging from “Yes, all of the time” to “No, never”; 4 items on 
experiences (e.g., hit/push others, spread lies about others, get help from 
another when having a bad time) with responses on 4-point scale of 
frequency ranging from “0 times” to “3 or more times.” 






X 


Knowledge: academic content 


Student direct assessment on state standardized achievement tests on reading 
and math. No additional information is provided. 




X 




Risk behaviors: crime, discipline 

issues 


Administrative data on criminal and discipline actions (e.g., disruptive, 
public lewdness). 




TABLE A. 35c 



PROGRAM NAME: TRIBES TLC 
Other Outcomes 



Level 


C onstruct/ Components 


Measurement Information 


School/Class Level 




School climate: collective norms, 
interactions 


Teacher report on student behavior among his/her students; 15 items on 
student behavior (e.g., listening, respecting each other) and engagement in 
learning (e.g., reflecting, being a leader); responses on 4-point scale of 
frequency (always to never) and on pattern (increased, decreased, same). 




School climate: collective norms, 
interactions 


Teacher report on the learning environment; 4 items on interview protocol 
concerning teacher-student interactions (e.g., recognize and reward student 
success, classroom rules determined together); responses asking for level of 
agreement, but no information provided on coding of interview data. 





TABLE A. 36a 



VOICES LITERATURE AND CHARACTER EDUCATION PROGRAM (V-LACE) 



Program name: 


Voices Literature and Character Education Program (V-LACE) 


Program type: 


Comprehensive or modular 


Grade level: 


K-12 (6-7 studied) 


Studies reviewed: 


Demetriades-Guyette, A. “Patterns of Change in the Social-Cognitive Development of 
Middle School Children Following a School-Based Multicultural Literature Program.” 
UMI No. 3052695. Dissertation Abstracts International, 2002. 


Description: 


Voices Literature and Character Education Program (V-LACE; formerly known as 
Voices of Love and Freedom and Literacy and Values) is a modular K-12 program that 
aims to promote positive character and citizenship values, literacy skills, and social skills. 
The program contains a curriculum that can be used over any length of time. During 
classroom lessons, students read books about such everyday issues as ethnic 
discrimination, fighting, and bullying, and elaborate on central themes through role- 
playing and discussions practiced in school and at home. Emphasis is given to promoting 
caring relationships between teachers and students and among students, and to connecting 
the values taught through students’ personal stories. V-LACE may also be implemented 
as a comprehensive, schoolwide improvement program. Optional components of the 
program include schoolwide events and restructuring of school organization and practices 
(establishing student assemblies and creating small learning communities), parental 
involvement (home visits and family nights), and community support (joint campaigns 
with supporting organizations and business). 
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TABLE A. 3 6b 



PROGRAM NAME: VOICES LITERATURE AND CHARACTER EDUCATION PROGRAM (V-LACE) 

Student Outcomes 



Affective 



X 



Behavioral Cognitive Constructs/Components 



Measurement Information 



X 



Competencies/skills: interpersonal 
competency 

Knowledge: interpersonal knowledge 



Student report on the Group for the Study of Interpersonal Development 
Relationship Questionnaire (GSID Rel-Q; Schultz et al. 2003), a measure of 
changes in social cognition that assesses the developmental level of social 
competence for children and adolescents in relationships with peers and adults. 
This version of the Rel-Q is a written, 24-item, multiple-choice questionnaire for 
children in grades 4-12. (A picture -based version is available for K-3.) This 
version of the Rel-Q measures relationship maturity through five subscales: (1) 
Interpersonal Understanding defined as one’s “knowledge of the nature of 
relationships” (6 items); (2) Interpersonal Negotiation, or one’s ability to manage 
conflicts in relationships (4 items); (3) Perspective Taking (4 items); (4) Conflict 
Negotiation Strategies (4 items); and (5) Personal Meaning, or the ability to 
evaluate one’s emotional investment in a relationship (6 items). Each multiple 
choice item presents a scenario or statement regarding relationships that requires 
a follow-up action or opinion; students must evaluate four potential follow-ups. 
Each option represents points in the continuum of four theoretical levels in the 
coordination of social perspectives, ranging from egocentric (Level 0), to 
unilateral (Level 1), to reciprocal (Level 2), to mutual (Level 3). Respondents rate 
whether each of the four follow-up options is “poor,” “OK,” “good” or 
“excellent” using a Likert-type scale, then indicating which of the four choices 
they believe is best. For example, one item states, “My parents are important to 
me because: (a) they make me feel better about myself; (b) they just are 
important; (c) they help me stay out of trouble; and (d) they provide the support 
that I need.” Scores are generated for both the “best choice” and item ratings. 



Prosocial: service 

Risk: violence, discipline issues 



Student self-reports were used to gauge the frequency of prosocial and risk 
behaviors in the past four weeks, including: (1) helping a peer; (2) physical fights 
in and (3) out of school; and (4) being sent to the principal’s office. 



X 




APPENDIX B 



CONSTRUCTS MEASURED FOR 36 SELECTED PROGRAMS 
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I. Student-Level Outcomes 
a. Cognitive 

i. Knowledge 

1. understanding values/norms 

2. risk prevention 

3. interpersonal knowledge 

4. intrapersonal knowledge 

5. academic content 

ii. Reasoning 

1. moral/ethical reasoning 

2. critical thinking/decision making 
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b. Affective 



Attitudes/Motives 

1. prosocial dispositions 

2. attitudes toward school 

3. attitudes toward risk/he2/lth 

4. civic dispositions 

5. attitudes toward diversity 

6. intrapersonal strengths 
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ii. Attitudes/Emotions 

1. caring (e.g., empathy) 

2. reflectivity 

3. school bonding/school engagement 

4. justice, fairness 
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c. Behavioral 



i. Competencies/Skills 

1. resistance 

V 

2. responsibility 

3. integrity V 

4. respect 

5. leadership 

6. intrapersonal competency 
(e.g., self-control) 

7. interpersonal competency 

8. communication 

9. coping 

ii. Prosocial Behaviors 

1. service ^ 

2. healthy lifestyle 

3. kindness 

4. trustworthiness 

5. justice, fairness 

6. positive participation 
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iii. 


Risk Behaviors 
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2. 


sexual risk-taking 
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violence 
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absence/tardiness 
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discipline issues 
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6. 


crime 
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II. 


Teacher-Admin Level Outcomes 




















a. 


Knowledge of child development 




















b. 


Support (endorse or value what goes on 
in school) 
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c. 


Understanding values/norms 
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Attendance 
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Staff morale V 
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f. Professional efficacy 



School-Level Outcomes 



a. School climate 

i. Collective n^rms/values 

ii. Interactions among students, staff, 
parents 

iii. Inclusion (e.g., individuals made to feel 

part of ti'fe school) ^ 

e. Social systems (e.g., advisories, 
leadership te^ims) 

f. Democratic governappe practices 

g. Positive leadership 
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h. Positive physical environment 

i. Positive academic environment 


V V V 












IV. Parent-Community Level Outcomes 


a. Parenting skills 
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b. Participation in school 
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c. Parent support of school/program 














d. Community climate/environm^nt 
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Notes: 

A a/ indicates that the relevant construct was addressed by at least one measure in one study of a particular program. Individual measures may reflect multiple 
constructs and multiple measures may be applied to measure the same construct within a single study. 

Due to limitations on available information, not all measures coidd be categorized at the lowest level of our taxonomy. Such constructs are reflected in the 
crosswalk by a checkmark in a row referring to one of the higher-levels of our taxonomy (e.g. “Behavioral”). Therefore, the construct counts presented in the 
text — which focus on the lowest level of the taxonomy — may be underestimates of the total number of programs that actually addressed that particidar construct. 
Constructs that could only be categorized at higher levels include the following: at the student-level, 2 measures were associated with reasoning, 1 with affect, 
11 with attitudes/motives, 5 with attitudes/emotions, 1 with behavior, 6 with competencies/skills, 11 with prosocial behaviors, and 8 with risk behaviors; at the 
school-level, one outcome was not categorized below the overarching category (school) and six measures of school climate could not be placed in the 
subcategories of collective norms/values, interactions, or inclusion. 
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